Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0007519opensim[REGION] OpenSim Corepublic2015-03-31 08:262015-10-14 15:34
ReporterEm Jannings 
Assigned To 
Platformdell computer x86 64 bitOperating SystemlinuxOperating System VersionUbuntu 14.04
Product Version 
Target VersionFixed in Version 
Summary0007519: OpenSim 0.8.2 dev83e58eb regions load and run for a few hours then crash with the following error.
DescriptionAPPLICATION EXCEPTION DETECTED: System.UnhandledExceptionEventArgs

Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at (wrapper unknown) System.Threading.Monitor:FastMonitorEnterV4 (object,bool&)
  at System.Threading.Timer+Scheduler.SchedulerThread () [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Application is terminating: True
Steps To Reproducesimply run the simulator and wait it always crashes within 24 hours
TagsNo tags attached.
Git Revision or version number0.8.2 dev 83e58eb
Run Mode Grid (Multiple Regions per Sim)
Physics EngineODE
Script Engine
EnvironmentMono / Linux64
Mono Version3.2
Attached Files

- Relationships

-  Notes
Allen Kerensky (reporter)
2015-03-31 08:58

Just some thinking out loud - there's a number of things you could try here:
In your try:

In OpenSim.ini try different settings for async_call_method.
I've had good results with QueueUserWorkItem as an alternative on Linux.

If you want to stick with SmartThreadPool (the usual best option for Linux) then you may try bumping up your MaxPoolThreads.
It defaults to 15 which was described as "good for Pentium 4" single core CPU in old comments.
Dual cores were commented as needing 30, and quad cores 45.
I've heard of people allocating 15-60 per core. Basically, try doubling it and seeing if the problem resolves or changes behavior.

Finally - try changing your Mono version.
3.2 is... newer than the stable version ( used for OpenSim build regression testing.
And 3.2 is older than the 3.12 currently available from Mono Project.
You might try downgrading to the stable regression version, or install the mono-project repo configuration for your base OS and update to current.

I am running OpenSim 0.8.2dev on Mono 3.12.1 on Fedora 21 latest patches without thread-related crashing.
Seth Nygard (reporter)
2015-04-01 20:31

I see you are running on linux with mono. I would strongly recommend you set the following settings;

AppDomainLoading = false
ScriptStopStrategy = co-op

It was discussed as one of the dev meetings towards the end of last year that when running under Linux, AppDomainLoading must be set to false. While this will contribute to some degree of memory bloat I generally don't see that problem to any significant degree.

I have been able to run regions for several weeks without crashes. I am running on Ubuntu 14.04LTS with mono 3.2.8

I would also look for problem scripts that may be running in the region(s) and tying up threads as that can be a cause for crashes.
Em Jannings (reporter)
2015-04-02 19:27

I did have AppDomainLoading = true -- even though I have a note that this is to be avoided on a linux server. I've set it to false and deleted all the .state files in ScriptEngines.
ScriptStopStrategy was set to co-op
I have been making it a practice to set MONO_THREADS_PER_CPU=150
and I am running Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1.1 I have another server with no problems using the same version.

These steps seem to have improved overall stability. I'm hoping the work to remove the .state script files will catch out any corrupt script when the simulator is restarted and the scripts are reset.

I'll report again after I have more time with these changes.

The comments are much appreciated.
Em Jannings (reporter)
2015-04-04 09:28
edited on: 2015-04-04 09:29

I think that the problem is probably not with the opensim software but rather with mono or my system. I do get runs of over 24 hours followed by abrupt crashes, I'll be trying to figure this out in the Ubuntu and mono forums. This is the latest:

08:11:56 - [SCENE]: Loaded 620 objects from the datastore

Native stacktrace:

    mono() [0x4b73d8]
    mono() [0x50f13b]
    mono() [0x423d22]
    /lib/x86_64-linux-gnu/ [0x7fda55e4f340]

Debug info from gdb:

Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No threads.

Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.

Aborted (core dumped)

smxy (reporter)
2015-04-04 10:57

I have this set in my environment:


I also run mono with the --server option (and the --debug option too).

Something else to try, anyway.
Orion Pseudo (reporter)
2015-04-21 18:31

Changing the default version of Mono that comes with Ubuntu to this one: [^]

Seems to have resolved the issue on my end.
Em Jannings (reporter)
2015-04-21 19:26

I backed off my version of Ubuntu OS from 14.04 to 12.04 and that has apparently resolved my problem. On another machine I'm running Lubuntu 14.04 and using the Metropolis version of the open simulator software without incident. I'm about to upgrade those servers to 0.8.2 and will be holding my breath when I start them up.
It's very helpful to get suggestions from this community about what one might try next. Thanks.
BlueWall (administrator)
2015-04-22 02:13

Look at this report to Mono, specifically the Linux kernel version [^]

typing uname -a in your shell will give you the kernel version.
Em Jannings (reporter)
2015-04-22 12:18

Thanks BlueWall. This is critical information. My regression to 12.04 gives me kernel 3.2.0-80 and no more crashes. So it appears I was just another example of this problem.

"Mono often crashes on newer Linux kernels.
Mono versions where this bug was tested and happens: 3.4, 3.8, 3.10
Linux kernels where Mono crashes: 3.14, 3.16
Linux kernels where Mono *doesn't* crash: 3.10, 3.11"
Em Jannings (reporter)
2015-04-22 12:32

This is apparently not a problem with open simulator, but a compatibility problem between certain versions of the linux kernel and versions of mono. A real fix requires either changes to the linux kernel or mono or both. see note from BlueWall below
nebadon (administrator)
2015-04-22 12:36

I don't want to set this to resolved, I have linked this to the mono bugzilla report from Oren, I don't want to give them the impression that its not a problem anymore.
Em Jannings (reporter)
2015-04-22 12:51

That makes sense to me -- I'm new at this so I thought I should take some responsibility for the disposition of my post. Obviously the problem is not fixed.
samuel.greenway (reporter)
2015-04-22 13:05
edited on: 2015-04-22 13:12

Ubuntu 14.04 LTS Desktop
Kernel 3.13.0-49-generic

My 2 cents. Im currently on Ubuntu 14.04 LTS and have recently starting having this issue. May have been there before, but now its not a once or twice a week occurrence, but several times a day.

I believe that Ubuntu server and desktop run same generic kernel, but my servers run 3.13.0-40-generic as the most recent.

Not currently running opensim on a server, but the desktop.

Only other kernel Ubuntu offers, that I'm aware of, is a low latency one.

EDIT: I did upgrade from stock mono 3.2.8 to mono 3.12.1 and it seemed to help, but still crashes, just not as often.

BlueWall (administrator)
2015-04-22 16:44

Here is a report I filed with OpenSuSE about the issue: [^]

If anyone using OpenSuSE 13.2 sees this it might be a good idea to file a report. They assigned the bug to me, but I think others monitor it.

User's of other distributions should file reports with the representative bug reporting channels they may have. It would be good to link these other reports to them showing the kernel versions affected in other distros, etc.
samuel.greenway (reporter)
2015-04-22 19:39

Reported with Ubuntu. I noted the other bug report links on this. [^]
There maybe duplicates, but I did search and did not find.

Had to revert to mono 3.2.8, since that is what comes with Ubuntu 14.04 LTS in order for Ubuntu to automatically start the report process.
Em Jannings (reporter)
2015-04-29 16:41

I see mono 4.0 is out. Does anyone know if it addresses this problem with linux?
sevo (reporter)
2015-05-05 13:20

Hi, I have similar problems with some completely different projects (simply found this report by googling for timer NREs) and I have reported it directly to Xamarin with a test application here: [^]

For me the problem with NREs in the timer scheduler thread occurs on different systems with different kernels (from 2.6.32 up to 3.19) and different mono versions (3.2 - 4.0.1) BUT only with SGEN gc. So maybe switching to BOEHM (--gc=boehm or simply mono-boehm binary) could help here, too.
samuel.greenway (reporter)
2015-05-05 19:41


mono --gc=boehm OpenSim.exe

running mono 3.12.1/Ubuntu 14.04.2

per sevo suggestion (see above) and will report back.
nebadon (administrator)
2015-05-05 19:45

you might need to use mono-boehm to launch I suspect --gc has been removed in newer versions of mono
samuel.greenway (reporter)
2015-05-05 20:28

As reported earlier, I was going to try mono --gc=boehm OpenSim.exe, but that did not seem to help. Since my regions keep crashing, I am now experience database corruption.
nebadon (administrator)
2015-05-05 20:31

please try using the command > mono-boehm OpenSim.exe < I am quite certain that -gc has been deprecated in mono 3.x+
samuel.greenway (reporter)
2015-05-05 21:00

Neb said ... "please try using the command > mono-boehm OpenSim.exe < I am quite certain that -gc has been deprecated in mono 3.x+"

It did start up with the switch -gc=boehm, but am trying the other way now. Will report back.
samuel.greenway (reporter)
2015-05-05 21:42
edited on: 2015-05-05 22:14

I used

mono-boehm OpenSim.exe

I also updated my .bashrc file to include

export MONO_GC_PARAMS=nursery-size=64m

And still get stacktrace error after running a user scheduled backup command.

...mono 3.12.1 / Ubuntu 14.04.2 / kernel 3.13.0-52-generic

Region (root) # backup
Triggering save of pending object updates to persistent store
Region (root) # Stacktrace:

Native stacktrace:

        mono() [0x4accac]
        mono() [0x50451f]
        mono() [0x42a7c7]
        /lib/x86_64-linux-gnu/ [0x7fd5c958f340]
        mono() [0x5c61c4]
        mono() [0x5cc577]
        mono() [0x5cc89d]
        mono() [0x5dfd05]
        mono() [0x5dffeb]

Debug info from gdb:

Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No threads.

Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.

Aborted (core dumped)

UPDATED..... still testing, as I realized I had 2 scripts running (one to start regions at boot, and one for crash recovery), but both did not have mono-boehm OpenSim.exe.

I had not installed the package that would allow mono-boehm to run. see next post.

samuel.greenway (reporter)
2015-05-05 22:16


On Ubuntu 14.04, I had previously upgraded from stock mono version 3.2.8 to 3.12.1, (see [^])

To run the command
  "mono-boehm OpenSim.exe",

I had to add package for mono-boehm
  "sudo apt-get install mono-runtime-boehm"

which in turn upgraded mono to version 4.0.1.

I had also added this to .bashrc, per Nebadon.
export MONO_GC_PARAMS=nursery-size=64m

I have not tried this on mono 3.2.8 which comes with Ubuntu 14.04LTS

Within a minute I got this error.... will monitor my regions for next few hours and report back.

Region (root) # Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Threading.Thread.SetName_internal (System.Threading.InternalThread,string) <0xffffffff>
  at System.Threading.Thread.set_Name (string) <0x00023>
  at OpenSim.Framework.Monitoring.WorkManager.StartThread (System.Threading.ThreadStart,string,System.Threading.ThreadPriority,bool,bool,System.Func`1<string>,int,bool) <0x000f7>
  at OpenSim.Framework.Monitoring.WorkManager.StartThread (System.Threading.ThreadStart,string,System.Threading.ThreadPriority,bool,bool,bool) <0x0004f>
  at OpenSim.Framework.Monitoring.WorkManager.RunInThread (System.Threading.WaitCallback,object,string,bool) <0x000f7>
  at OpenSim.Region.Framework.Scenes.Scene.UpdateStorageBackup () <0x000a3>
  at OpenSim.Region.Framework.Scenes.Scene.Update (int) <0x00c2f>
  at OpenSim.Region.Framework.Scenes.Scene.Heartbeat () <0x0020d>
  at System.Threading.Thread.StartInternal () <0x0006f>
  at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0xffffffff>

Native stacktrace:

        mono-boehm() [0x4b202c]
        mono-boehm() [0x5085fe]
        mono-boehm() [0x428ebd]
        /lib/x86_64-linux-gnu/ [0x7f3d44ec3340]
        /lib/x86_64-linux-gnu/ [0x7f3d44ec4432]
        mono-boehm() [0x5f94da]
        mono-boehm() [0x58416b]

Debug info from gdb:

Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
No threads.

Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.

Aborted (core dumped)
nebadon (administrator)
2015-05-05 22:31

ok it is likely the kernel then, unfortunate!
samuel.greenway (reporter)
2015-05-05 22:56

My 11 instances (some running single regions, 1 varable, and some running multiple regions) ran without error for about 40 mins after my initial error. I did see a memory usage increase by about 20%. I shutdown my regions to compile opensim under newer mono 4.0.1, and received errors during the compile and was not able to complete. Will troubleshoot more later.
sevo (reporter)
2015-05-06 00:04

Hhm, as I have written using boehm is a workaround for the NRE in the timer (see initial bug description). I have no problems with crashes in native code (e.g. SIGSEGV). So there could be different upstream bugs.
nebadon (administrator)
2015-05-06 06:52

sevo are you talking directly about OpenSimulator or something else? OpenSimulator uses a crazy amount of threads.
sevo (reporter)
2015-05-06 07:49

No, I have no idea about OpenSimulator. I simply commented here, cause the initial description is about the Timer and I have the same problems with timers in other private projects. I think that there are two different problems, one with the timer scheduler loosing some pointers with SGen (see my bug report at Xamarin) and the other one with excessive thread usage. So if you have problems with timers try BOEHM. The threading problem seems to be solved at least for me with mono 4.0.
samuel.greenway (reporter)
2015-05-06 10:43
edited on: 2015-05-06 10:44

After leaving my regions running overnight (past 12 hours) I have had no further crashes, other than my initial crash at startup on 1 out of my 11 instances, reported above. I do see a slight memory increase with my total memory usage (including operating system) going from around 5.5gb - 6gb to around 6.5gb - 7gb.

Current setup ...

Ubuntu 14.04.2
kernel 3.13.0-52-generic
Mono 4.0.1, mono-complete mono-runtime-boehm

in .bashrc ...
export MONO_GC_PARAMS=nursery-size=64m

Will continue to monitor.

May revert to stock Ubuntu and mono (using mono-boehm) at some point and report back.

samuel.greenway (reporter)
2015-05-06 22:42

Update, after 24 hours, still getting stacktrace errors, but not as often as I was. I was getting multiple crashes an hour, now only every few hours.
andreapiero arbizu (reporter)
2015-05-08 10:01

Check start mono with option --server ( mono --server .... ).
AliciaRaven (manager)
2015-05-13 09:10

In the post above "" [^] the crash occurs after the log line "Loaded x objects from the datastore". I had been getting a SIGSEGV at that same point during start up. Not every time but enough to be annoying. After I increased the MONO_THREADS_PER_CPU the errors stopped.

However, at first i still hadn't set the value high enough, so at one point OS crashed at that same point during start up, but this time it wasn't as severe and OS managed to print out an error message (Shown Below).

I'm wondering if this is what is causing the crashes, and because most cases OS goes down before it has chance to print an error, the problem could not be identified.

Maybe this can be fixed so that OS doesn't take down mono when it hits the problem or it may just be a coincidence that this happened at the same point.


2015-05-12 02:21:15,537 INFO - OpenSim.Region.Framework.Scenes.Scene [SCENE]: Loading objects from datastore
2015-05-12 02:21:19,071 INFO - OpenSim.Region.Framework.Scenes.Scene [SCENE]: Loaded 463 objects from the datastore
2015-05-12 02:21:20,063 INFO - OpenSim.Region.Framework.Scenes.Scene [SCENE]: This region has 4,945 prims
2015-05-12 02:21:20,130 INFO - OpenSim.Region.Framework.Scenes.Scene [SCENE]: Loading land objects from storage
2015-05-12 02:21:24,285 ERROR - OpenSim.Application [APPLICATION]:
APPLICATION EXCEPTION DETECTED: System.UnhandledExceptionEventArgs

Exception: System.NullReferenceException: Object reference not set to an instance of an object
  at System.Collections.Generic.List`1+Enumerator[OpenSim.Region.Physics.BulletSPlugin.BSScene+TaintCallbackEntry].MoveNext () [0x00000] in <filename unknown>:0
  at OpenSim.Region.Physics.BulletSPlugin.BSScene.ProcessRegularTaints () [0x00000] in <filename unknown>:0
  at OpenSim.Region.Physics.BulletSPlugin.BSScene.ProcessTaints () [0x00000] in <filename unknown>:0
  at OpenSim.Region.Physics.BulletSPlugin.BSScene.DoPhysicsStep (Single timeStep) [0x00000] in <filename unknown>:0
  at OpenSim.Region.Physics.BulletSPlugin.BSScene.BulletSPluginPhysicsThread () [0x00000] in <filename unknown>:0
  at System.Threading.Thread.StartInternal () [0x00000] in <filename unknown>:0

Application is terminating: True
samuel.greenway (reporter)
2015-05-14 11:18

Update on my setup.

Seems export MONO_THREADS_PER_CPU=2048 helped me the most.
I reverted back to mono 3.2.8 from 4.0.1 and see no difference.

With both, I still get occasional (1-4 a day) crashes.

Im using a lot more memory than I would like and have had to cut back on my regions. (I have a core2quad 2.4, 8gb of ram, and try to keep all my regions on that one system)
samuel.greenway (reporter)
2015-05-14 11:20


Received an email from the bug reporter and he advised to...
"try compile the master- next branch of trusty to see if it fixes the issue for OpenSim" [^]
Boater (reporter)
2015-05-17 01:47

I am suffering from this bug too, hard for me as I was boasting to one in America that Ubuntu stays up and don't need a reboot unlike windoze
Shy Robbiani (reporter)
2015-05-22 03:11
edited on: 2015-05-22 03:14

According to my many tests under various configurations and investigations it seems the crashes only happen on 64-bit systems!

In my case I experienced and can reproduce the problem at any time with Mono 3.2.8 as well as Mono 4.0.1 under the following systems: Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-52-generic x86_64), Ubuntu 14.04.2 LTS (GNU/Linux 3.16.0-37-generic x86_64) and Debian 8 (3.16.0-4-amd64).

I tested on 64-bit VM guests only (Oracle Virtualbox and Bochs). Under similar conditions Mono 4.0.1 seems to crash more often and faster than Mono 3.2.8.

I had not yet the chance to test in a 32-bit Linux environment, but from what I heard from others running 32-bit Linux there is no crash under a similar combination of software releases.

Shy Robbiani (reporter)
2015-05-23 17:11

After many tests under various conditions I found out that the source of the problem is in the Linux kernel. I don't know which kernel release provides a fix for the problem but with the latest stable Linux kernel 4.0.4 the problem no longer exists. The problem is neither directly related to Opensim nor to Mono.

In addition I wrote a simple threading application to reproduce and demonstrate the problem. I've been running several tests and none of them failed with Linux kernel v4.0.4.
samuel.greenway (reporter)
2015-05-23 20:55

According to Ubuntu bug report: [^]

New kernel version ending in -54 (which is pending release) seems to have this issue fixed.
Also its noted on that bug report that this only effects 64bit versions and not 32bit versions. I have not verified any of this information.
nebadon (administrator)
2015-05-23 20:56

I have tested on OpenSuSE Tumbleweed with mono 4.0.1 and Kernel 4.0.3 and it still is crashing. Just for the record.
Shy Robbiani (reporter)
2015-05-26 09:26

Sorry, I was reporting here too early. In my configuration with Mono 4.0.1 and Kernel 4.0.4 it still crashes, it takes just much longer.
samuel.greenway (reporter)
2015-06-10 22:40
edited on: 2015-06-10 22:41

Currently testing a new Ubuntu kernel (3.13.0-54-generic) and it seems to run good.

See report: [^]

I am currently running:
Ubuntu 14.04.2
mono 3.2.8 (stock)
ubuntu kernel 3.13.0-54-generic (newly released kernel)
Intel Core2Quad Q6600 2.4ghz
8 gb of RAM

I am not using:
export MONO_GC_PARAMS=nursery-size=64m
I am using whatever mono uses (stock) out of the box for testing.

I have 11 simulators running with 40 regular regions (about 10 have signification prim counts), 1 6x6 variable region.
One of the regular regions has around 1900 listener scripts.

All seems to be functioning properly. Inworld seems stable and smooth under normal use. They have been running over an hour, with no crashes. Before I would have had a half dozen to a dozen crashes under the above conditions (minus new kernel).

I will continue to monitor and report back.

Em Jannings (reporter)
2015-06-11 10:37
edited on: 2015-06-11 10:39

I'm also running the 3.13.0-54-generic kernel (Lubuntu 14.04) on a NUC with an i5 quad core and 16 GB of RAM, Mono 3.2.8, THREADS_PER_CPU set to 150.

I'm using the Metropolis release of OpenSim ( metropolis-os_082_DEV_PLUS [017]) running seven regions, some with fairly heavy loads. I've been running for 20 days without a crash.

Boater Burt (reporter)
2015-06-26 07:45

I am running the following With Ubuntu and it still crashes regularly.

mike@boater:~$ uname -a
Linux boater 3.13.0-55-generic 0000094-Ubuntu SMP Thu Jun 18 00:28:41 UTC 2015 i686 i686 i686 GNU/Linux

mike@boater:~$ mono --version
Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1.1)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors.
    TLS: __thread
    SIGSEGV: altstack
    Notifications: epoll
    Architecture: x86
    Disabled: none
    Misc: softdebug
    LLVM: supported, not enabled.
    GC: sgen

I tried:
export MONO_GC_PARAMS=nursery-size=64m
and it makes no difference
Boater Burt (reporter)
2015-06-26 07:58

Don't know where 0000094 came from but its hash_sign94 on my screen.
Boater Burt (reporter)
2015-07-05 00:44

No answers? It's been over a week now, I still reset my Diva about twice a day,
and OSG less often, though it needs resetting, average every two days.
Boater Burt (reporter)
2015-07-17 23:19

It's still crashing at the same rate, even though I updated. I now run

 Linux boater 3.13.0-57-generic 0000095-Ubuntu SMP Fri Jun 19 09:27:48 UTC 2015 i686 i686 i686 GNU/Linux

The Mono reports the same version. Toying now with purging mono and reinstalling.
samuel.greenway (reporter)
2015-10-14 14:51
edited on: 2015-10-14 15:33

I believe the fix in ubuntu kernel -54 was reverted upstream (correction, better to say fix was removed from later kernels after -54), at some point.

Looks like the origional bug report was closed and new one opened with Ubuntu, but not much activity.

Old: [^]

New: [^]

I have not had any recent issues, but I have my grub set to load only -54 kernel. I have had reports of others still experiencing this under Ubuntu 14.04 x64 with stock mono version.

Em Jannings (reporter)
2015-10-14 15:06
edited on: 2015-10-14 15:24

Noob question: Does "reverted upstream" mean that this kernel fix has been propagated to earlier kernel versions? And does that effectively eliminate this bug? Thanks.

samuel.greenway (reporter)
2015-10-14 15:34
edited on: 2015-10-14 15:35

correction, better for me to say fix the fix implemented in -54 was removed from kernels sometime after -54 (Ubuntu)

- Issue History
Date Modified Username Field Change
2015-03-31 08:26 Em Jannings New Issue
2015-03-31 08:58 Allen Kerensky Note Added: 0027956
2015-04-01 20:31 Seth Nygard Note Added: 0027957
2015-04-02 19:27 Em Jannings Note Added: 0027965
2015-04-04 09:28 Em Jannings Note Added: 0027969
2015-04-04 09:29 Em Jannings Note Edited: 0027969 View Revisions
2015-04-04 09:29 Em Jannings Note Edited: 0027969 View Revisions
2015-04-04 10:57 smxy Note Added: 0027970
2015-04-21 18:31 Orion Pseudo Note Added: 0028047
2015-04-21 19:26 Em Jannings Note Added: 0028048
2015-04-22 02:13 BlueWall Note Added: 0028049
2015-04-22 12:18 Em Jannings Note Added: 0028051
2015-04-22 12:20 nebadon Status new => confirmed
2015-04-22 12:32 Em Jannings Note Added: 0028052
2015-04-22 12:32 Em Jannings Status confirmed => resolved
2015-04-22 12:32 Em Jannings Resolution open => fixed
2015-04-22 12:32 Em Jannings Assigned To => Em Jannings
2015-04-22 12:36 nebadon Note Added: 0028053
2015-04-22 12:36 nebadon Assigned To Em Jannings =>
2015-04-22 12:36 nebadon Status resolved => acknowledged
2015-04-22 12:37 nebadon Assigned To => nebadon
2015-04-22 12:37 nebadon Status acknowledged => assigned
2015-04-22 12:37 nebadon Assigned To nebadon =>
2015-04-22 12:37 nebadon Description Updated View Revisions
2015-04-22 12:37 nebadon Status assigned => acknowledged
2015-04-22 12:51 Em Jannings Note Added: 0028054
2015-04-22 13:05 samuel.greenway Note Added: 0028056
2015-04-22 13:06 samuel.greenway Note Edited: 0028056 View Revisions
2015-04-22 13:12 samuel.greenway Note Edited: 0028056 View Revisions
2015-04-22 16:44 BlueWall Note Added: 0028061
2015-04-22 19:39 samuel.greenway Note Added: 0028062
2015-04-29 16:41 Em Jannings Note Added: 0028142
2015-05-05 13:20 sevo Note Added: 0028221
2015-05-05 19:41 samuel.greenway Note Added: 0028224
2015-05-05 19:45 nebadon Note Added: 0028225
2015-05-05 20:28 samuel.greenway Note Added: 0028226
2015-05-05 20:31 nebadon Note Added: 0028227
2015-05-05 21:00 samuel.greenway Note Added: 0028228
2015-05-05 21:42 samuel.greenway Note Added: 0028231
2015-05-05 21:47 samuel.greenway Note Edited: 0028231 View Revisions
2015-05-05 22:14 samuel.greenway Note Edited: 0028231 View Revisions
2015-05-05 22:16 samuel.greenway Note Added: 0028232
2015-05-05 22:31 nebadon Note Added: 0028233
2015-05-05 22:56 samuel.greenway Note Added: 0028234
2015-05-06 00:04 sevo Note Added: 0028235
2015-05-06 06:52 nebadon Note Added: 0028237
2015-05-06 07:49 sevo Note Added: 0028238
2015-05-06 10:43 samuel.greenway Note Added: 0028250
2015-05-06 10:44 samuel.greenway Note Edited: 0028250 View Revisions
2015-05-06 22:42 samuel.greenway Note Added: 0028264
2015-05-08 10:01 andreapiero arbizu Note Added: 0028308
2015-05-11 03:51 Shy Robbiani Note Added: 0028361
2015-05-11 04:13 Shy Robbiani Note Deleted: 0028361
2015-05-13 09:10 AliciaRaven Note Added: 0028367
2015-05-14 11:18 samuel.greenway Note Added: 0028376
2015-05-14 11:20 samuel.greenway Note Added: 0028377
2015-05-17 01:47 Boater Note Added: 0028386
2015-05-22 03:11 Shy Robbiani Note Added: 0028425
2015-05-22 03:13 Shy Robbiani Note Edited: 0028425 View Revisions
2015-05-22 03:14 Shy Robbiani Note Edited: 0028425 View Revisions
2015-05-23 17:11 Shy Robbiani Note Added: 0028439
2015-05-23 20:55 samuel.greenway Note Added: 0028440
2015-05-23 20:56 nebadon Note Added: 0028441
2015-05-26 09:26 Shy Robbiani Note Added: 0028449
2015-06-10 22:40 samuel.greenway Note Added: 0028655
2015-06-10 22:41 samuel.greenway Note Edited: 0028655 View Revisions
2015-06-11 10:37 Em Jannings Note Added: 0028656
2015-06-11 10:39 Em Jannings Note Edited: 0028656 View Revisions
2015-06-26 07:45 Boater Burt Note Added: 0028822
2015-06-26 07:58 Boater Burt Note Added: 0028823
2015-07-05 00:44 Boater Burt Note Added: 0028865
2015-07-17 23:19 Boater Burt Note Added: 0028918
2015-10-14 14:51 samuel.greenway Note Added: 0029506
2015-10-14 14:57 samuel.greenway Note Edited: 0029506 View Revisions
2015-10-14 14:59 samuel.greenway Note Edited: 0029506 View Revisions
2015-10-14 15:06 Em Jannings Note Added: 0029507
2015-10-14 15:24 Em Jannings Note Edited: 0029507 View Revisions
2015-10-14 15:33 samuel.greenway Note Edited: 0029506 View Revisions
2015-10-14 15:34 samuel.greenway Note Added: 0029508
2015-10-14 15:35 samuel.greenway Note Edited: 0029508 View Revisions

Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker