[Opensim-users] Unusual MySql Timeouts - Ferd Frederix

[Fred]

A user of stock Opensim (Robust or Standalone) 0.9.0 or 0.8.20 is having 
unusual MySQL timeouts that I cannot explain or fix.  The timeout 
happens when the BackupWorker thread kicks in to flush the data to 
disk.  So it also happens when I manually type in 'backup' and it 
happens when I type 'quit', too, both also trigger the backup.

Mysql:  She is running Mysql 5.5 on a dedicated server.  The 
max_allowed_packet = 128M, and the database passes all mysqlupdate and 
mysqlrepairs on mysql, robust and opensim databases with no errors.

The prims or item it fails on seem to be random.  As a result of the 
backup thread aborting, items are always missing on the region after the 
restart.   Nothing goes missing unless she shuts down and restarts.

One region loses the same part of the same house each time it reboots, 
another region lose more than half the prims. This is somewhat 
consistent.  But the same oar loads on other machines and works fine. It 
is something with this particular machine.

Originally it seems to be due to a bad hard disk, but that has since 
been replaced.  It happens on Server 2016, and on Windows 10.  The 
server has 32 GB of RAM, an I7 CPU

The error will occur with just one of many regions when loaded with an 
oar. The oars are in the 30K-45,000 prim range.

Any ideas on what to do?

16:53:52 - Timeout in threadfunc 13094 (STP:Util:40) (BackupWorker (Town 
of Haven))    at System.Net.UnsafeNclNativeMethods.OSSOCK.recv(IntPtr 
socketHandle, Byte* pinnedBuffer, Int32 len, SocketFlags socketFlags)
    at System.Net.UnsafeNclNativeMethods.OSSOCK.recv(IntPtr 
socketHandle, Byte* pinnedBuffer, Int32 len, SocketFlags socketFlags)
    at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, 
Int32 size, SocketFlags socketFlags, SocketError& errorCode)
    at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 
offset, Int32 size)
    at MySql.Data.Common.MyNetworkStream.Read(Byte[] buffer, Int32 
offset, Int32 count)
    at MySql.Data.MySqlClient.TimedStream.Read(Byte[] buffer, Int32 
offset, Int32 count)
    at System.IO.BufferedStream.Read(Byte[] array, Int32 offset, Int32 
count)
    at MySql.Data.MySqlClient.MySqlStream.ReadFully(Stream stream, 
Byte[] buffer, Int32 offset, Int32 count)
    at MySql.Data.MySqlClient.MySqlStream.LoadPacket()
    at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
    at MySql.Data.MySqlClient.NativeDriver.GetResult(Int32& affectedRow, 
Int64& insertedId)
    at MySql.Data.MySqlClient.Driver.NextResult(Int32 statementId, 
Boolean force)
    at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
    at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior 
behavior)
    at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
    at 
OpenSim.Data.MySQL.MySQLSimulationData.ExecuteNonQuery(MySqlCommand c) 
in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Data\MySQL\MySQLSimulationData.cs:line 
116
    at 
OpenSim.Data.MySQL.MySQLSimulationData.StoreObject(SceneObjectGroup obj, 
UUID regionUUID) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Data\MySQL\MySQLSimulationData.cs:line 
262
    at 
OpenSim.Services.SimulationService.SimulationDataService.StoreObject(SceneObjectGroup 
obj, UUID regionUUID) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Services\SimulationService\SimulationDataService.cs:line 
84
    at 
OpenSim.Region.Framework.Scenes.SceneObjectGroup.ProcessBackup(ISimulationDataService 
datastore, Boolean forcedBackup) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Region\Framework\Scenes\SceneObjectGroup.cs:line 
2544
    at 
OpenSim.Region.Framework.Scenes.EventManager.TriggerOnBackup(ISimulationDataService 
dstore, Boolean forced) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Region\Framework\Scenes\EventManager.cs:line 
1410
    at OpenSim.Region.Framework.Scenes.Scene.Backup(Boolean forced) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Region\Framework\Scenes\Scene.cs:line 
1983
    at 
OpenSim.Region.Framework.Scenes.Scene.<UpdateStorageBackup>b__1f(Object 
o) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Region\Framework\Scenes\Scene.cs:line 
1946
    at 
OpenSim.Framework.Util.<>c__DisplayClass9.<FireAndForget>b__2(Object o) 
in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Framework\Util.cs:line 
2675
    at OpenSim.Framework.Util.<FireAndForget>b__3(WaitCallback cb, 
Object o) in 
c:\Opensim\Outworldz-Source\OutworldzFiles\opensim\OpenSim\Framework\Util.cs:line 
2761
    at 
Amib.Threading.Internal.WorkItemsGroupBase.<>c__DisplayClass7`2.<QueueWorkItem>b__6(Object 
state) in c:\Opensim\Opensim 0.9.0 
release\opensim\ThirdParty\SmartThreadPool\WorkItemsGroupBase.cs:line 333
    at Amib.Threading.Internal.WorkItem.ExecuteWorkItem() in 
c:\Opensim\Opensim 0.9.0 
release\opensim\ThirdParty\SmartThreadPool\WorkItem.cs:line 379
    at Amib.Threading.Internal.WorkItem.Execute() in c:\Opensim\Opensim 
0.9.0 release\opensim\ThirdParty\SmartThreadPool\WorkItem.cs:line 312
    at Amib.Threading.SmartThreadPool.ExecuteWorkItem(WorkItem workItem) 
in c:\Opensim\Opensim 0.9.0 
release\opensim\ThirdParty\SmartThreadPool\SmartThreadPool.cs:line 882
    at Amib.Threading.SmartThreadPool.ProcessQueuedItems() in 
c:\Opensim\Opensim 0.9.0 
release\opensim\ThirdParty\SmartThreadPool\SmartThreadPool.cs:line 822
    at System.Threading.ExecutionContext.RunInternal(ExecutionContext 
executionContext, ContextCallback callback, Object state, Boolean 
preserveSyncCtx)
    at System.Threading.ExecutionContext.Run(ExecutionContext 
executionContext, ContextCallback callback, Object state, Boolean 
preserveSyncCtx)
    at System.Threading.ExecutionContext.Run(ExecutionContext 
executionContext, ContextCallback callback, Object state)
    at System.Threading.ThreadHelper.ThreadStart()

After the above warning,  it throws an error on items like this:

13:51:49 - [REGION DB]: MySQL error in ExecuteNonQuery: Thread was being 
aborted.
13:51:49 - [SCENE]: Storing of salon-StylistCenter Stand&Drawers v1, 
15f57bde-610b-43a9-97a1-946b31a72273 in Town of Haven failed with 
exception Thread was being aborted. at 
MySql.Data.Common.MyNetworkStream.HandleOrRethrowException(Exception e)
at MySql.Data.Common.MyNetworkStream............

Fred Beckhusen/Ferd Frederix
www.outworldz.com
fred at outworldx.com