LLUDP ClientStack
From OpenSimulator
(Difference between revisions)
(→Inbound UDP) |
|||
| (22 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| + | {{Quicklinks|LLUDP_ClientStack}} | ||
| + | |||
=Introduction= | =Introduction= | ||
| Line 16: | Line 18: | ||
=Useful console commands= | =Useful console commands= | ||
| + | |||
| + | Note these console commands are no longer available in OpenSimulator. The description below is left in place for historical information. | ||
These are console commands which are useful in debugging or investigating the LLUDP protocol. | These are console commands which are useful in debugging or investigating the LLUDP protocol. | ||
| − | * debug lludp packet [--default] <level> [<avatar-first-name> <avatar-last-name>] - Turn on packet debugging. In OpenSimulator 0.7.5 and previous this command was "debug packet". | + | * debug lludp packet [--default | --all] <level> [<avatar-first-name> <avatar-last-name>] - Turn on packet debugging. In OpenSimulator 0.7.5 and previous this command was "debug packet". "--default" applies the new logging level to all avatar that login after that point. "--all" applies it to all existing avatars as well. |
* debug lludp pool <on|off> - Turn object pooling within the lludp component on or off. | * debug lludp pool <on|off> - Turn object pooling within the lludp component on or off. | ||
* debug lludp start <in|out|all> - Control LLUDP packet processing. | * debug lludp start <in|out|all> - Control LLUDP packet processing. | ||
| Line 25: | Line 29: | ||
* debug lludp stop <in|out|all> - Stop LLUDP packet processing. | * debug lludp stop <in|out|all> - Stop LLUDP packet processing. | ||
| − | = | + | =Inbound UDP= |
| − | + | == Code paths == | |
| − | |||
===Part 1=== | ===Part 1=== | ||
* Inbound UDP messages are received on a region's listening UDP port (currently in OpenSimUDPBase). These are IOCP threads from the runtime as processing is done asynchronously. | * Inbound UDP messages are received on a region's listening UDP port (currently in OpenSimUDPBase). These are IOCP threads from the runtime as processing is done asynchronously. | ||
| Line 39: | Line 42: | ||
** Is CompleteAgentMovement (used to complete the insertion of an avatar into a region) then this is handled on a separate thread. | ** Is CompleteAgentMovement (used to complete the insertion of an avatar into a region) then this is handled on a separate thread. | ||
** Has appended acks to a normal packet or is PacketAck then these are processed. | ** Has appended acks to a normal packet or is PacketAck then these are processed. | ||
| − | ** Is reliable then a pending ack is queued for | + | ** Is reliable then a pending ack is queued for acknowledgement to the client. |
** Is AgentUpdate and is not significant (same as last packet, where these are received about 10 times a second) then it is discarded. | ** Is AgentUpdate and is not significant (same as last packet, where these are received about 10 times a second) then it is discarded. | ||
** StartPingCheck then this is handled. | ** StartPingCheck then this is handled. | ||
| Line 49: | Line 52: | ||
* It enters the LLClientView structure, logging details of the packet for debug purposes if required. | * It enters the LLClientView structure, logging details of the packet for debug purposes if required. | ||
* The packet is processed either synchronously (on this same thread) or asynchronously by queueing it as a work item to a threadpool (SmartThreadPool by default). | * The packet is processed either synchronously (on this same thread) or asynchronously by queueing it as a work item to a threadpool (SmartThreadPool by default). | ||
| + | ** Some packets are handled synchronously due to issues if they are processed out of order (e.g. multi-face texture changes not occuring properly). Others may be synchronous by default where they could actually be handled asynchronously. | ||
* The long running thread loops. | * The long running thread loops. | ||
| + | |||
| + | =Outbound UDP= | ||
| + | |||
| + | == Code paths == | ||
| + | '''This section is very incomplete.''' | ||
| + | * LLUDPServer.SendPacketFinal() is the penultimate method before UDP data is placed on the wire. | ||
| + | ** If the packet is not zero-coded and has room, then any waiting acks are appended. | ||
| + | ** If the packet is reliable then the server records its expectation of acks from the client. | ||
| + | |||
| + | * If there are no packets waiting to be sent for a throttle and none in the outbox, then BeginFireQueueEmpty(). | ||
| + | |||
| + | * LLUDPClient.BeingFireQueueEmpty fires HasUpdates event is fired to determine if updates are waiting. | ||
| + | ** LLClientView.HandleHasUpdates() handles this and returns true if it has queued entity updates, entity property updates or image manager updates (UDP asset download). | ||
| + | |||
| + | * If there are such updates waiting, a threadpool FireQueueEmpty() thread is triggered | ||
| + | * This then fires the OnQueueEmpty event. | ||
| + | ** Which LLClientView handles with HandleQueueEmpty() which places a certain number of updates via OutPacket() | ||
| + | |||
| + | == Throttles == | ||
| + | '''There is also an old informal discussion of throttles at [[Sim Throttles]] which may or may not be accurate.''' | ||
| + | |||
| + | Outbound UDP is throttled per client, with an optional throttle for the entire scene. | ||
| + | |||
| + | === Settings === | ||
| + | |||
| + | If | ||
| + | |||
| + | <source lang="ini"> | ||
| + | [ClientStack.LindenUDP] | ||
| + | enable_adaptive_throttles = true | ||
| + | </source> | ||
| + | |||
| + | (the current default). Then client throttles are adjusted automatically by the server. Whilst the connection is fine, rates are increased. However, if packets marked as reliable are not received in a timely manner, then the throttle is reduced until no more packets are lost. The throttle is then gradually increased again until either the throttle requested by the client is reached or more packets are lost. | ||
| + | |||
| + | If | ||
| + | |||
| + | <source lang="ini"> | ||
| + | [ClientStack.LindenUDP] | ||
| + | enable_adaptive_throttles = false | ||
| + | </source> | ||
| + | |||
| + | Then throttles are set by the client. | ||
| + | |||
| + | In dependent of adaptive or client throttle settings, throttle rates may still be restricted on the server side by setting either or both of | ||
| + | |||
| + | <source lang="ini"> | ||
| + | [ClientStack.LindenUDP] | ||
| + | client_throttle_max_bps = <some-client-limit> | ||
| + | scene_throttle_max_bps = <some-entire-scene-limit> | ||
| + | </source> | ||
| + | |||
| + | The client_throttle_max_bps setting will enforce a maximum limit for each client connection. The scene_throttle_max_bps setting will set a maximum output limit for all connections to a scene. So for this latter limit, every extra connection will reduce the bandwidth for everybody as a fixed limit is shared between more clients. This includes child connections (used for clients with avatars in neighbouring regions to see activity in this region). | ||
| + | |||
| + | Neither of these limits are set by default. | ||
=Other references= | =Other references= | ||
* [[Sim Throttles]] contains very old information on the implementation of throttles. This has likely changed considerably but could still be useful. | * [[Sim Throttles]] contains very old information on the implementation of throttles. This has likely changed considerably but could still be useful. | ||
Latest revision as of 04:01, 4 December 2023
Contents |
[edit] Introduction
These are draft notes on OpenSimulator's implementation of the LLUDP ClientStack, a chunk of code which is used to send and receive packets from viewers (clients) implementing the Linden Labs virtual environment protocol.
As such, this stack handles
- Setup of an inbound UDP handling stack for each region.
- Handling of inbound UDP messages from a connected viewer.
- Distribution of recieved UDP messages to appropriate handling code (e.g. selection handling code if a prim is selected).
- Sending of outbound UDP messages to a connected viewer.
- Throttling of outbound messages.
- Sending and receive of ack messages, both inline within other messages and as standalone messages.
- Resending of messages that are marked as reliable but for which receipt has not been acknowledged by the viewer.
- Throttling of outgoing UDP messages.
- Pooling of clientstack structures (e.g. classes representing messages) in order to improve efficiency and reduce memory usage.
[edit] Useful console commands
Note these console commands are no longer available in OpenSimulator. The description below is left in place for historical information.
These are console commands which are useful in debugging or investigating the LLUDP protocol.
- debug lludp packet [--default | --all] <level> [<avatar-first-name> <avatar-last-name>] - Turn on packet debugging. In OpenSimulator 0.7.5 and previous this command was "debug packet". "--default" applies the new logging level to all avatar that login after that point. "--all" applies it to all existing avatars as well.
- debug lludp pool <on|off> - Turn object pooling within the lludp component on or off.
- debug lludp start <in|out|all> - Control LLUDP packet processing.
- debug lludp status - Return status of LLUDP packet processing.
- debug lludp stop <in|out|all> - Stop LLUDP packet processing.
[edit] Inbound UDP
[edit] Code paths
[edit] Part 1
- Inbound UDP messages are received on a region's listening UDP port (currently in OpenSimUDPBase). These are IOCP threads from the runtime as processing is done asynchronously.
- On receive, the thread kicks off another asychrnous receive (which may be handled by a different IOCP thread).
- Each received packet is sanity checked for length and basic malformations.
- Data is inserted into a packet structure drawn from a pool.
- If packet
- Is UseCircuitCode (used to set up circuits with viewers) then this is handled on a separate thread.
- Is CompleteAgentMovement (used to complete the insertion of an avatar into a region) then this is handled on a separate thread.
- Has appended acks to a normal packet or is PacketAck then these are processed.
- Is reliable then a pending ack is queued for acknowledgement to the client.
- Is AgentUpdate and is not significant (same as last packet, where these are received about 10 times a second) then it is discarded.
- StartPingCheck then this is handled.
- All other packets are placed into a queue (packetInbox) for further processing.
- The thread is released (synchronous fetch not currently operational).
[edit] Part 2
- A continuously running thread fetches the next packet at the front of the packetInbox queue.
- It enters the LLClientView structure, logging details of the packet for debug purposes if required.
- The packet is processed either synchronously (on this same thread) or asynchronously by queueing it as a work item to a threadpool (SmartThreadPool by default).
- Some packets are handled synchronously due to issues if they are processed out of order (e.g. multi-face texture changes not occuring properly). Others may be synchronous by default where they could actually be handled asynchronously.
- The long running thread loops.
[edit] Outbound UDP
[edit] Code paths
This section is very incomplete.
- LLUDPServer.SendPacketFinal() is the penultimate method before UDP data is placed on the wire.
- If the packet is not zero-coded and has room, then any waiting acks are appended.
- If the packet is reliable then the server records its expectation of acks from the client.
- If there are no packets waiting to be sent for a throttle and none in the outbox, then BeginFireQueueEmpty().
- LLUDPClient.BeingFireQueueEmpty fires HasUpdates event is fired to determine if updates are waiting.
- LLClientView.HandleHasUpdates() handles this and returns true if it has queued entity updates, entity property updates or image manager updates (UDP asset download).
- If there are such updates waiting, a threadpool FireQueueEmpty() thread is triggered
- This then fires the OnQueueEmpty event.
- Which LLClientView handles with HandleQueueEmpty() which places a certain number of updates via OutPacket()
[edit] Throttles
There is also an old informal discussion of throttles at Sim Throttles which may or may not be accurate.
Outbound UDP is throttled per client, with an optional throttle for the entire scene.
[edit] Settings
If
[ClientStack.LindenUDP] enable_adaptive_throttles = true
(the current default). Then client throttles are adjusted automatically by the server. Whilst the connection is fine, rates are increased. However, if packets marked as reliable are not received in a timely manner, then the throttle is reduced until no more packets are lost. The throttle is then gradually increased again until either the throttle requested by the client is reached or more packets are lost.
If
[ClientStack.LindenUDP] enable_adaptive_throttles = false
Then throttles are set by the client.
In dependent of adaptive or client throttle settings, throttle rates may still be restricted on the server side by setting either or both of
[ClientStack.LindenUDP] client_throttle_max_bps = <some-client-limit> scene_throttle_max_bps = <some-entire-scene-limit>
The client_throttle_max_bps setting will enforce a maximum limit for each client connection. The scene_throttle_max_bps setting will set a maximum output limit for all connections to a scene. So for this latter limit, every extra connection will reduce the bandwidth for everybody as a fixed limit is shared between more clients. This includes child connections (used for clients with avatars in neighbouring regions to see activity in this region).
Neither of these limits are set by default.
[edit] Other references
- Sim Throttles contains very old information on the implementation of throttles. This has likely changed considerably but could still be useful.
