Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0008392opensim[REGION] OpenSim Corepublic2018-10-06 23:272020-04-04 08:16
ReporterCodyCooper 
Assigned To 
PrioritynormalSeveritymajorReproducibilitysometimes
StatusconfirmedResolutionopen 
PlatformOSOS Version
Product Version0.9.0.1 
Target VersionFixed in Version 
Summary0008392: When Using MySQL 5.8.X (latest) , Simulator will fail to store assets
DescriptionSeveral People, including I, have experienced where the simulator logs a sql error, saying "x077x44x454...etc" is not a proper format for field "text", this causes several assets to not persist.

However upon importing database to a MySQL Server with 5.7.23, Not Only is the Persistence Problem fixed, but the Simulator Performance is Improved, Loading Times seem to improve , and region crossings are almost Perfect.

This only seems to occur when storing high ascii characters
TagsNo tags attached.
Git Revision or version number
Run Mode Grid (Multiple Regions per Sim)
Physics EngineBulletSim
Script Engine
Environment.NET / Windows64
Mono VersionNone
ViewerFirestorm
Attached Files

- Relationships

-  Notes
(0033159)
BillBlight (developer)
2018-10-10 23:09

Was able to replicate on my test grid, and with talking to others with the same error.
(0033168)
Luisillo_Contepomi (reporter)
2018-10-15 04:18

Please what opensimulator version are you talking about?
For me mytsql latest version is MySQL 8.0.12
I am testing mysql 8.0.12 with opensim 0.9.0.1 (dev included) and I have not sql errors and seem all is running perfectly.
(0033169)
BillBlight (developer)
2018-10-15 09:17

Luisillo_Contepomi this is with the 9x branch and only occurs when storing high ascii charaters
(0033176)
CodyCooper (reporter)
2018-10-18 10:17

watcher64 , this occurs even with simple letters and words like SF Pig
(0033177)
CodyCooper (reporter)
2018-10-18 10:18

It is not just with the OpenSim 9x branch, but also with 0.8.2.1 at least.
(0036307)
Ferd Frederix (reporter)
2020-03-30 14:12

Not an Opensim bug. The database needs to be set to use utf-8.

For My.ini on Windows:

[client]
default-character-set = utf8

[mysql]
default-character-set = utf8


[mysqld]
character-set-server=utf8
character-set-client-handshake = FALSE

See https://mathiasbynens.be/notes/mysql-utf8mb4 [^] for details on the 4 byte changes necessary to support all possible Unicode characters.
(0036308)
tampa (reporter)
2020-03-30 15:25

Charset can be set for tables individually and according to the migrations for Assets is set to utf8 so I suspect there is either something else going wrong or the syntax for setting the charset has perhaps changed so the migration is not executed properly. That is if the charset is the actual cause of the issue.

The ticket is now over a year old, has anyone got a recent confirmation this is still occurring?
(0036309)
djphil (reporter)
2020-03-30 17:00
edited on: 2020-03-30 17:28

In addition I added this in My.ini:

[mysqld]
collation-server = utf8_general_ci

PS: I'm not sure with utf8 we can properly store all emoji's unlike utf8mb4.
https://getemoji.com/ [^]

Already several special characters are a pain when displayed on a dynamic texture or a viewer menu window.
http://opensimulator.org/mantis/view.php?id=8095 [^]
http://opensimulator.org/mantis/view.php?id=8192 [^]

(0036320)
Ferd Frederix (reporter)
2020-03-31 13:10

This is still occurring in Satyr farm, see https://github.com/SatyrFarm/SatyrFarm/issues/16 [^]
(0036321)
tampa (reporter)
2020-03-31 13:53
edited on: 2020-03-31 14:38

mysql 5.8 isn't a thing, 5.7 is previous stable, current stable is 8.0, so which is it actually?

Reading the release notes for 8 it states clearly that collation and charset names are different and specifically utf8 has been changed. Has anyone tried altering the table to those new names?

Setting collation as default for the entire mysql server should not be necessary as databases and tables are independent and, as the name suggests, you are only setting defaults, which are applied if the desired collation or charset are not available to the server, which points toward the migrations. If changing the charset and collation of the table to the new names for utf8 and the problem goes away, then it is trivial to add one with a #version conditional to upgrade them based on the mysql version used.

Edit: Upon further reading there are even more changes that potentially break some of the rather loose syntax OpenSim uses to handle data so mysql 8 may entirely reject statements if older syntax is used that, even back then, isn't proper sql standard. Either don't upgrade or use MariaDB; latter works just as well if not better and is more forgiving of the sloppy OpenSim syntax.

(0036330)
djphil (reporter)
2020-04-04 02:14
edited on: 2020-04-04 02:35

Mobius Grid's work seems to confirm the need to use utf8mb4 instead of utf8 for the support of special characters and probably emoji's too.

See Beta 1.3 Notes @ https://github.com/Mobius-Team/Mobius-Public/tree/beta#beta-13-notes [^]

(0036339)
tampa (reporter)
2020-04-04 08:16

As I said, setup tables with proper collation and charset and this should fix it, may need to upgrade migrations at some point to reflect changes in MySQL 8, but MariaDB is still an option and I would wager to say a better option over MySQL.

- Issue History
Date Modified Username Field Change
2018-10-06 23:27 CodyCooper New Issue
2018-10-06 23:30 CodyCooper Severity crash => major
2018-10-06 23:30 CodyCooper Description Updated View Revisions
2018-10-10 23:09 BillBlight Note Added: 0033159
2018-10-10 23:09 BillBlight Status new => confirmed
2018-10-15 04:18 Luisillo_Contepomi Note Added: 0033168
2018-10-15 09:16 BillBlight Product Version => 0.9.0.1
2018-10-15 09:17 BillBlight Note Added: 0033169
2018-10-15 09:17 BillBlight Description Updated View Revisions
2018-10-18 10:17 CodyCooper Note Added: 0033176
2018-10-18 10:18 CodyCooper Note Added: 0033177
2020-03-30 14:12 Ferd Frederix Note Added: 0036307
2020-03-30 15:25 tampa Note Added: 0036308
2020-03-30 17:00 djphil Note Added: 0036309
2020-03-30 17:16 djphil Note Edited: 0036309 View Revisions
2020-03-30 17:17 djphil Note Edited: 0036309 View Revisions
2020-03-30 17:27 djphil Note Edited: 0036309 View Revisions
2020-03-30 17:28 djphil Note Edited: 0036309 View Revisions
2020-03-30 17:28 djphil Note Edited: 0036309 View Revisions
2020-03-31 13:10 Ferd Frederix Note Added: 0036320
2020-03-31 13:53 tampa Note Added: 0036321
2020-03-31 14:38 tampa Note Edited: 0036321 View Revisions
2020-04-04 02:14 djphil Note Added: 0036330
2020-04-04 02:28 djphil Note Edited: 0036330 View Revisions
2020-04-04 02:35 djphil Note Edited: 0036330 View Revisions
2020-04-04 08:16 tampa Note Added: 0036339


Copyright © 2000 - 2012 MantisBT Group
Powered by Mantis Bugtracker