lotrointerface.com
Search Downloads


Go Back   LoTROInterface > LotRO > Developer Discussions > Lua Programming Help (L)

Reply
Thread Tools Display Modes
  #1  
Unread 11-06-2010, 12:10 PM
Chiran Chiran is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Posts: 27
Plugin support for German and French clients

Since there's not yet discussion about this, I thought I'd start one.

So what we have is a nasty bug with how data is serialized when using the internal Turbine.PluginData.Save method. What happens is that with German (and French?) clients the default decimal separator, which is "." (dot), gets converted to "," (comma). This can obviously break the save file badly since it's parsed as a Lua file. We can expect that this will be fixed by Turbine in the future, but plugin authors can react to this quite a bit faster. So, what should we do about it?

1) Each and every plugin author should find their own way to adjust plugins so that they work for all client versions. That is, if they want or bother to (it can be a lot of work).
2) We expect that those using other clients than English will start to use tools such as the fix provided by Vindar.

Personally I'd like to modify mine so that they do work for everyone. But then again does that interfere with the fix? I've also found it problematic to test this since it doesn't appear that anything short of installing a German client is enough to make this bug appear.

-Chiran

Last edited by Chiran : 11-06-2010 at 01:03 PM.
Reply With Quote
  #2  
Unread 11-06-2010, 06:13 PM
Pengoros Pengoros is offline
The Unscathed
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Posts: 15
If you alias the Turbine.PluginData.Load and Turbine.PluginData.Save methods it will cause the fix to not change anything in your plugin like this:

Code:
local load_alias = Turbine.PluginData.Load;
local save_alias = Turbine.PluginData.Save;

-- now just use the alias functions to do the saving or loading
save_alias(Turbine.DataScope.Character, "PluginName", object_to_save);
local loaded_object = load_alias(Turbine.DataScope.Character, "PluginName");
Reply With Quote
  #3  
Unread 11-06-2010, 06:55 PM
Chiran Chiran is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Posts: 27
Ah, that's a good tip, thanks!

-Chiran
Reply With Quote
  #4  
Unread 11-06-2010, 07:48 PM
Digital_Utopia's Avatar
Digital_Utopia Digital_Utopia is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Posts: 207
Send a message via MSN to Digital_Utopia Send a message via Yahoo to Digital_Utopia
Since Palantir has a hefty number of settings, I was planning on creating a couple of functions to iterate through the tables before saving, and after loading to convert numbers/bool to strings and back.
__________________

Lord of the Rings Online
75 Fourohfour | 75 Artemedis | 60 Whiskeytango Foxtrot | 50 Mistah Boombastic | 56 Appetizer | 25 Aggromi
61 Onepointtwentyone Gigawatts


World of Warcraft
90 Downlo 85 Gravetaxi 85 Ümad 85 Artemedis 85 Guthuros
Reply With Quote
  #5  
Unread 11-06-2010, 10:02 PM
NuclearTonic's Avatar
NuclearTonic NuclearTonic is offline
The Undefeated
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Location: Rochester, NY
Posts: 8
Quote:
Originally Posted by Digital_Utopia
Since Palantir has a hefty number of settings, I was planning on creating a couple of functions to iterate through the tables before saving, and after loading to convert numbers/bool to strings and back.
That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.
Reply With Quote
  #6  
Unread 11-07-2010, 09:08 AM
Digital_Utopia's Avatar
Digital_Utopia Digital_Utopia is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: Sep 2010
Posts: 207
Send a message via MSN to Digital_Utopia Send a message via Yahoo to Digital_Utopia
Quote:
Originally Posted by NuclearTonic
That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.
It might just be because I don't have the TurbinePlugins folder anymore, but I couldn't find that particular code. Instead, I just went with these functions:

Code:
function settingsEncode(t)
	local t2={};
	for k,v in pairs(t) do
		if(type(v)=="number")then
			v=tostring(v);
		elseif (type(v)=="table")then
			v=settingsEncode(v);
		end
		t2[k]=v;
	end
	return t2
end

Code:
function settingsDecode(t)
	local t2={};
	for k,v in pairs(t) do
		if(tonumber(v)~=nil)then
			v=tonumber(v);
		elseif (type(v)=="table")then
			v=settingsDecode(v);
		end
		t2[k]=v;
	end
	return t2
end
__________________

Lord of the Rings Online
75 Fourohfour | 75 Artemedis | 60 Whiskeytango Foxtrot | 50 Mistah Boombastic | 56 Appetizer | 25 Aggromi
61 Onepointtwentyone Gigawatts


World of Warcraft
90 Downlo 85 Gravetaxi 85 Ümad 85 Artemedis 85 Guthuros
Reply With Quote
  #7  
Unread 05-20-2013, 11:36 PM
Thurallor's Avatar
Thurallor Thurallor is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: May 2013
Posts: 202
Post

I implemented a slightly different solution for my plugin.

Basically, before I call Turbine.PluginData.Save(), I serialize my settings table (and subtables) into a string. The string is simply a snippet of Lua code that, when executed, returns my settings table. Then I save that single string with Turbine.PluginData.Save().

After I get the string back from Turbine.PluginData.Load(), I use the built-in loadstring() function to parse it back into a table.

Vindar's fix didn't work for me, because my settings table contains tables that look like this:

Code:
t = { "a", "b", "c" }
and Vindar's loader was returning them like this:

Code:
t = {
   ["1"] = "a",
   ["2"] = "b",
   ["3"] = "c"
}
Unfortunately, there is a difference. I hacked Vindar's code for a while, but I couldn't get it to work. So I implemented my alternative. Here's the source code:
Code:
function SaveSettings(settings)
    -- Serialize (workaround for Turbine localization bug)
    local saveStr = "return " .. Serialize(settings);
    Turbine.PluginData.Save(Turbine.DataScope.Character, "Stuff", saveStr, function()
        -- Finished saving
    end);
end

function LoadSettings()
    Turbine.PluginData.Load(Turbine.DataScope.Character, "Stuff", function(saveStr)
        if (saveStr) then
            -- Unserialize (workaround for Turbine localization bug)
            settings = assert(loadstring(saveStr))();
            if (not settings) then
                Turbine.Shell.WriteLine("Failed to parse Stuff.plugindata!");
            end
        end
    end);
end

function Serialize(obj)
    if (type(obj) == "nil") then
        return "nil";
    elseif (type(obj) == "boolean") then
        if (obj) then
            return "true";
        else
            return "false";
        end
    elseif (type(obj) == "number") then
        local text = tostring(obj);
        -- Change floating-point numbers to English format
        return string.gsub(text, ",", ".");
    elseif (type(obj) == "string") then
        return string.format("%q", obj);
    elseif (type(obj) == "table") then
        local text = "{";
        for i, v in pairs(obj) do
            local index = Serialize(i);
            local value = Serialize(v);
            if (value ~= nil) then
                local item = "[" .. index .. "]=" .. value .. ",";
                text = text .. item;
            end
        end
        text = string.gsub(text, ",$", "");
        text = text .. "}";
        return text;
    else
        return nil;
    end
end
You may notice the only escaping I'm doing for strings is replacing " with \" and \ with \\. I'm not sure if there's anything else I need to do, such as for strange foreign characters. Can anyone advise me?

Last edited by Thurallor : 05-25-2013 at 09:16 PM. Reason: Now using the built-in string.format("%q", ...) facility
Reply With Quote
  #8  
Unread 05-22-2013, 06:39 PM
Equendil Equendil is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: Apr 2011
Posts: 52
Quote:
Originally Posted by Thurallor
You may notice the only escaping I'm doing for strings is replacing " with \" and \ with \\. I'm not sure if there's anything else I need to do, such as for strange foreign characters. Can anyone advise me?
'Foreign' characters are encoded into UTF8. While the lua interpreter does not handle that encoding, lua strings are nothing more than arbitrary sequences of bytes, and they are transmitted to and fro the game engine and file system without change, so you generally don't need to worry about it for storing/retrieving strings.

Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.

Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"
__________________
Author of LIP, Bootstrap and Baruk

Last edited by Equendil : 05-22-2013 at 06:48 PM.
Reply With Quote
  #9  
Unread 05-22-2013, 07:26 PM
Thurallor's Avatar
Thurallor Thurallor is offline
The Undying
Interface Author - Click to view interfaces
 
Join Date: May 2013
Posts: 202
Quote:
Originally Posted by Equendil
'Foreign' characters are encoded into UTF8. While the lua interpreter does not handle that encoding, lua strings are nothing more than arbitrary sequences of bytes, and they are transmitted to and fro the game engine and file system without change, so you generally don't need to worry about it for storing/retrieving strings.
Hi Equendil, thanks for the reply. I use your table browser plugin quite a lot.

Quote:
Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.
Hmm, good point. And I do have some strings containing linefeeds. May explain some flaky behavior I've been seeing.

Edit: Maybe I can avoid the whole problem by using "long-bracket" literal strings: [[ ... ]].

Edit: Or by using string.format("%q", ...). Updated the source code.

Quote:
Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"
That's just me being anal. It turns

t = { "a", "b", }

into

t = { "a", "b" }

even though the former is completely acceptable in Lua. For the sake of maximizing efficiency, perhaps I should remove it.

Last edited by Thurallor : 05-25-2013 at 09:16 PM. Reason: afterthought
Reply With Quote
  #10  
Unread 06-10-2013, 11:03 AM
Lynx3d Lynx3d is offline
The Undefeated
Interface Author - Click to view interfaces
 
Join Date: May 2013
Posts: 7
Seems like I won't get around this stuff either...

Thurallor, I can't reproduce the problem of Vindar's code returning number indices as string indices. Are you sure you converted the saved data back?
But in any case it only solves the locale problem partially, it will still save numbers as "1,234" and hence still fails converting back at loading when switching client language.
Also I'm not sure why he needs that metatable and other "black magic", I'm pretty sure references to the same table will be lost anyway.

Thurallor's solution is certainly interesting, it basically does the same as Turbine's own I/O method by converting it to Lua-code. Only problem is that you need to wrap the result in a single huge string which makes pretty unreadable files, and I'm not too sure that you won't eventually run into any line-length limitation of the interpreter or underlying file I/O.

I decided to mix the solutions, I just write out numbers as string and parse them back with loadstring(), and prefix the strings so know if it was was number or not:
Code:
function convSave(obj)
	if type(obj) == "number" then
		local text = "n:" .. tostring(obj);
		return string.gsub(text, ",", ".");
	elseif type(obj) == "string" then
		return "s:" .. obj;
	elseif type(obj) == "boolean" then
		return obj;
	elseif type(obj) == "table" then
		local newt = {};
		for k, v in pairs(obj) do
			if type(k) ~= "number" and type(k) ~= "string" then
				-- unsupported keys
			else
				newt[convSave(k)] = convSave(v);
			end
		end
		return newt;
	end
end

function convLoad(obj)
	if type(obj) == "string" then
		obj_id = string.sub(obj, 1, 2);
		if obj_id == "n:" then
			-- need to run it through interpreter, since tonumber() may only accept ","
			local readnum = loadstring("return " .. string.sub(obj, 3));
			return readnum and readnum();
		elseif obj_id == "s:" then
			return string.sub(obj, 3)
		end
	elseif type(obj) == "boolean" then
		return obj;
	elseif type(obj) == "table" then
		local newt = {}
		for k, v in pairs(obj) do
			local key = convLoad(k);
			if key ~= nil then
				newt[key] = convLoad(v);
			end
		end
		return newt;
	end
end
So a table like
{ [5] = 1.23, ["foo"] = "bar"}
becomes
{ ["n:5"] = "n:1.23", ["s:foo"] = "s:bar" }
which can be converted back in all locales.

I could also check for current locale and use gsub again and use tonumber() instead of loadstring() for numbers, and I haven't investigated if Lotro can handle non-printable characters or if I should use string.format("%q", ...) too...
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Better plugin management Olenn LotRO Wish List (L) 6 10-21-2010 12:09 AM
Plugin Manager 0.2 help jingo386 Script & Skin Managers 1 09-20-2010 11:33 PM
Plugin Manager Sirak Script & Skin Managers 8 09-12-2010 04:35 PM
Lua support coming to LOTRO Sparr General Authoring Discussion (L) 33 08-05-2010 07:52 PM
Lua support coming to LOTRO's UI! Dolby News 1 07-15-2010 09:36 AM


All times are GMT -5. The time now is 01:39 PM.


Our Network
EQInterface | EQ2Interface | Minion | WoWInterface | ESOUI | LoTROInterface | MMOUI | Swtorui