Sim Throttles

Revision as of 13:39, 28 March 2011

 

Home

 

Download

 

News

 

Support

 

Admin

 

Dev

 

Screen Shots

 

Grid List

 

Bugs

Languages:

English

Technical Reference -> Terms -> Sim Throttles

Technical Reference

What are Sim Throttles?

Protocol throttling discussion by Teravus Ousley from #opensim-dev

D.. just a note.. I've gotten a few questions about how the throttle I put in a few revisions ago works and I think this channel can benefit from the description also.

Basically the throttle's purpose is to send only so much data to the client at one time. Too much data and the client drops packets and objects/textures don’t rez / are gray, too little data and things load excruciatingly slow. We don’t want to flood the client and we don’t want them to wait too long.

Previously, we've *set* the throttler max value to various levels over the course of the project.

Last it was 256kbps The new throttler however, doesn't work that way.

In clientview.cs there are some minimum and maximum throttle settings that you can set The previous throttler had a maximum outgoing data amount only the new throttler has 8 different throttles. Therefore it also has a Minimum outgoing bytes per second, a Maximum outgoing bytes per second, and a default outgoing bytes per second *for each throttle* The other thing that this throttler does differently is 'listen to what the client asks for the throttles to be' .. that's the reason for the Bytes per second Minimum We don't want to be throttling on the sim endlessly if the client says "Hey send me 0 bytes of data on Land"

The reason for the 8 different types is because of the way things work in this world :D.. Each type of data has it's own throttle settings because there are times when, for example, a billion textures are being downloaded that it's necessary to send 'other' information too The other information might include; updates about where other people are and updates about where you are. We can't let that huge influx of texture data cause the sim to not send you updates about where 'you' are otherwise, you'd freeze in place until all that texture data is downloaded

Therefore I split all of the outgoing traffic into types The types are: Resend, Land, Task, wind, clouds, texture, and asset types. There's also the total outgoing maximum throttle. That makes 8 different throttles

The description of the throttles types in opensim are as follows:

Resend: When a packet is sent with a reliable flag, it must be acknowledged by the client. The client needs to send back an 'ack' packet within an allotted amount of time. If the client doesn't send back an ack packet, the packet gets resent.

Land: Any land layer related data. Any heightfield updates.. etc.

Wind: Any changes in the wind data of the sim.

Clouds: Any changes in the cloud layer data.

Texture: Any and all textures.

Asset: Any asset type transactions.. like Downloading your inventory.. for example. and finally 'Task:'

Task is sort of.. any agent, prim, or event going on in the sim besides land, wind, clouds, texture, asset.. etc.

Now, I didn't come up with this data scheme typing.. Linden Labs did :D. And they programmed the client to request different throttles for each of these types It sends them using the 'AgentThrottle' packet

But, I digress…

The packet throttler has a minimum, maximum and default outgoing throttle setting for each of these types and a 'total outgoing' throttle. The default values in clientview.cs are 'reasonable' but I expect that they will be tweaked in the future.

Fair warning: This 'speech' gets more 'geeky' from here. . The UDP system has an endless ClientLoop function that has what's known as a blocking queue. It uses a blockingqueue to keep it from consuming all of the sim's resources. The blocking queue basically stops the loop while there's no traffic coming in or out. This blocking queue, in the code is called PacketQueue The important thing to know about it, is when there's no data in the queue, it stops the endless clientloop function.

Now.. since we're accepting input from the client about what the network throttles should be.. we need to make sure they are sane. The minimum and maximum values are for that purpose.

If the client requests an outgoing throttle (for any of the types) that's lower then the minimum, then the client will get the minimum throttle amount that we set.

If the client requests an outgoing throttle( for any of the types) that's greater then the maximum that we set, then the client will get the maximum throttle amount that we set.

If the total amount of all of the throttle values is over the maximum total outgoing throttle amount that we set, I scale the values down by their percent of the total amount of Bytes per second the sim will scale the throttles down in the amount equal to the percentage of the total connection the client requested.

One example, lets say the maximum outbound throttle is set at 1000000kbps, and the client requests the following throttles: 200000-land, 100000-wind, 500000-task, 100000-cloud,300000-texture, 100000 - asset, 300000 - Resend. The total Bytes Per second is 1,600,000. This is over the maximum throttle so I take a percentage for each type of the total connection and apply it to a 1000000kbps maximum. land's percentage of the total is 12.5%, wind is 6.25%, etc.. Then I apply that percentage to the maximum outbound throttle kbps to get a scaled throttle value

The only one that doesn't get scaled is the Resend. It's done that way because scaling or otherwise modifying what the client requests there would be bad. Just make sure it's sane and under the maximum throttle for the outboundresend throttle the throttle also works by dumping the packets (if we're over the throttle values) into queues that empty out into the main blockingqueue a couple of times a second if need be up to the throttle limit. This doesn't compete with current traffic because at that point current traffic is going into the queues anyway. Once again, there's a queue for each type of traffic and there's a routine to make sure that the throttle for each type of traffic is also respected. The different throttle settings also makes sure that when we're throttling, we send the maximum packets possible under the throttle so the queue doesn't continue to build up to infinitum. If we've got 90% of the traffic being 'texture' related traffic and 10% land, map, task, etc. Then, the texture traffic will be throttled, but the land map task and other packets will fill up the maximum available outbound space.

In conclusion, the new packet throttler takes input from the client about what the client wants the throttles to be set and keeps the throttles set at sane values.

To change the maximum amount of Bytes Per second that your sim will send out, you need only to change the maximum outgoing throttle bytes per second setting.

If you don't change them, the amount of data going out could range between 5000 bytes per second to 1540000 bytes per second depending on what the client has set in the 'network' pane of preferences and how many prim/textures/avatar are in the sim at the same time.

In practice the main burst of traffic happens when the client logs in after that, the majority of traffic is about textures.

Another thing the client tries to 'self heal' by changing throttling settings based on it's experience of the network conditions if it experiences lots of packet loss, it throttles down all by itself.

Maybe a better analogy is a 6 Lane outgoing highway where each type of vehicle has a designated lane (Land,Wind,Clouds,Texture,Asset,Task) and one for emergencies(resend). Each lane also has a limit of vehicles that can travel it at a time. Every vehicle over the limit has to wait in traffic. The client asks for a certain amount of vehicles of each. The simulator also has it's idea of the minimum and the maximum number of vehicles that the client can ask for per lane and overall. If the client asks for too much, the simulator doesn't tell the client no. It simply adds up the vehicle limits for all of lanes asked for by the client. It figures out each individual lane's percent of the added up vehicles in the last step, then uses the simulator's maximum number of vehicles total and the individual lane percentage of the whole in the request to get the new lane vehicle max. If the client asks for too little, then the simulator sets it at it's lower bound. Each lane has it's capacity. That's the maximum. Each lane also has a minimum so that a client can't simply 'stop' traffic on one lane causing it to back up, road rage and a highway shooting. When set reasonably, this works. When not set reasonably, this could cause real issues. Lets say the client asked for 10,000,000bps task and 1,000bps for all of the other lanes but the simulator maximum is set for 1,000,000bps. You can see where this is going. I think the minimum is applied after the percentage so that, at least protects the simulator from stupid but the minimum doesn't usually have a good experience. The simulator has minimums and maximums for each lane. It also has an overall maximum.