LoTROInterface

LoTROInterface (https://www.lotrointerface.com/forums/index.php)
-   Lua Programming Help (L) (https://www.lotrointerface.com/forums/forumdisplay.php?f=50)
-   -   Plugin support for German and French clients (https://www.lotrointerface.com/forums/showthread.php?t=1195)

Chiran 11-06-2010 12:10 PM

Plugin support for German and French clients
 
Since there's not yet discussion about this, I thought I'd start one.

So what we have is a nasty bug with how data is serialized when using the internal Turbine.PluginData.Save method. What happens is that with German (and French?) clients the default decimal separator, which is "." (dot), gets converted to "," (comma). This can obviously break the save file badly since it's parsed as a Lua file. We can expect that this will be fixed by Turbine in the future, but plugin authors can react to this quite a bit faster. So, what should we do about it?

1) Each and every plugin author should find their own way to adjust plugins so that they work for all client versions. That is, if they want or bother to (it can be a lot of work).
2) We expect that those using other clients than English will start to use tools such as the fix provided by Vindar.

Personally I'd like to modify mine so that they do work for everyone. But then again does that interfere with the fix? I've also found it problematic to test this since it doesn't appear that anything short of installing a German client is enough to make this bug appear.

-Chiran

Pengoros 11-06-2010 06:13 PM

If you alias the Turbine.PluginData.Load and Turbine.PluginData.Save methods it will cause the fix to not change anything in your plugin like this:

Code:

local load_alias = Turbine.PluginData.Load;
local save_alias = Turbine.PluginData.Save;

-- now just use the alias functions to do the saving or loading
save_alias(Turbine.DataScope.Character, "PluginName", object_to_save);
local loaded_object = load_alias(Turbine.DataScope.Character, "PluginName");


Chiran 11-06-2010 06:55 PM

Ah, that's a good tip, thanks!

-Chiran

Digital_Utopia 11-06-2010 07:48 PM

Since Palantir has a hefty number of settings, I was planning on creating a couple of functions to iterate through the tables before saving, and after loading to convert numbers/bool to strings and back.

NuclearTonic 11-06-2010 10:02 PM

Quote:

Originally Posted by Digital_Utopia (Post 5431)
Since Palantir has a hefty number of settings, I was planning on creating a couple of functions to iterate through the tables before saving, and after loading to convert numbers/bool to strings and back.

That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.

Digital_Utopia 11-07-2010 09:08 AM

Quote:

Originally Posted by NuclearTonic (Post 5434)
That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.

It might just be because I don't have the TurbinePlugins folder anymore, but I couldn't find that particular code. Instead, I just went with these functions:

Code:

function settingsEncode(t)
        local t2={};
        for k,v in pairs(t) do
                if(type(v)=="number")then
                        v=tostring(v);
                elseif (type(v)=="table")then
                        v=settingsEncode(v);
                end
                t2[k]=v;
        end
        return t2
end


Code:

function settingsDecode(t)
        local t2={};
        for k,v in pairs(t) do
                if(tonumber(v)~=nil)then
                        v=tonumber(v);
                elseif (type(v)=="table")then
                        v=settingsDecode(v);
                end
                t2[k]=v;
        end
        return t2
end


Thurallor 05-20-2013 11:36 PM

I implemented a slightly different solution for my plugin.

Basically, before I call Turbine.PluginData.Save(), I serialize my settings table (and subtables) into a string. The string is simply a snippet of Lua code that, when executed, returns my settings table. Then I save that single string with Turbine.PluginData.Save().

After I get the string back from Turbine.PluginData.Load(), I use the built-in loadstring() function to parse it back into a table.

Vindar's fix didn't work for me, because my settings table contains tables that look like this:

Code:

t = { "a", "b", "c" }
and Vindar's loader was returning them like this:

Code:

t = {
  ["1"] = "a",
  ["2"] = "b",
  ["3"] = "c"
}

Unfortunately, there is a difference. :) I hacked Vindar's code for a while, but I couldn't get it to work. So I implemented my alternative. Here's the source code:
Code:

function SaveSettings(settings)
    -- Serialize (workaround for Turbine localization bug)
    local saveStr = "return " .. Serialize(settings);
    Turbine.PluginData.Save(Turbine.DataScope.Character, "Stuff", saveStr, function()
        -- Finished saving
    end);
end

function LoadSettings()
    Turbine.PluginData.Load(Turbine.DataScope.Character, "Stuff", function(saveStr)
        if (saveStr) then
            -- Unserialize (workaround for Turbine localization bug)
            settings = assert(loadstring(saveStr))();
            if (not settings) then
                Turbine.Shell.WriteLine("Failed to parse Stuff.plugindata!");
            end
        end
    end);
end

function Serialize(obj)
    if (type(obj) == "nil") then
        return "nil";
    elseif (type(obj) == "boolean") then
        if (obj) then
            return "true";
        else
            return "false";
        end
    elseif (type(obj) == "number") then
        local text = tostring(obj);
        -- Change floating-point numbers to English format
        return string.gsub(text, ",", ".");
    elseif (type(obj) == "string") then
        return string.format("%q", obj);
    elseif (type(obj) == "table") then
        local text = "{";
        for i, v in pairs(obj) do
            local index = Serialize(i);
            local value = Serialize(v);
            if (value ~= nil) then
                local item = "[" .. index .. "]=" .. value .. ",";
                text = text .. item;
            end
        end
        text = string.gsub(text, ",$", "");
        text = text .. "}";
        return text;
    else
        return nil;
    end
end

You may notice the only escaping I'm doing for strings is replacing " with \" and \ with \\. I'm not sure if there's anything else I need to do, such as for strange foreign characters. Can anyone advise me?

Equendil 05-22-2013 06:39 PM

Quote:

Originally Posted by Thurallor (Post 8635)
You may notice the only escaping I'm doing for strings is replacing " with \" and \ with \\. I'm not sure if there's anything else I need to do, such as for strange foreign characters. Can anyone advise me?

'Foreign' characters are encoded into UTF8. While the lua interpreter does not handle that encoding, lua strings are nothing more than arbitrary sequences of bytes, and they are transmitted to and fro the game engine and file system without change, so you generally don't need to worry about it for storing/retrieving strings.

Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.

Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"

Thurallor 05-22-2013 07:26 PM

Quote:

Originally Posted by Equendil (Post 8642)
'Foreign' characters are encoded into UTF8. While the lua interpreter does not handle that encoding, lua strings are nothing more than arbitrary sequences of bytes, and they are transmitted to and fro the game engine and file system without change, so you generally don't need to worry about it for storing/retrieving strings.

Hi Equendil, thanks for the reply. I use your table browser plugin quite a lot. :)

Quote:

Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.
Hmm, good point. And I do have some strings containing linefeeds. May explain some flaky behavior I've been seeing.

Edit: Maybe I can avoid the whole problem by using "long-bracket" literal strings: [[ ... ]].

Edit: Or by using string.format("%q", ...). Updated the source code.

Quote:

Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"
That's just me being anal. It turns

t = { "a", "b", }

into

t = { "a", "b" }

even though the former is completely acceptable in Lua. For the sake of maximizing efficiency, perhaps I should remove it.

Lynx3d 06-10-2013 11:03 AM

Seems like I won't get around this stuff either...

Thurallor, I can't reproduce the problem of Vindar's code returning number indices as string indices. Are you sure you converted the saved data back?
But in any case it only solves the locale problem partially, it will still save numbers as "1,234" and hence still fails converting back at loading when switching client language.
Also I'm not sure why he needs that metatable and other "black magic", I'm pretty sure references to the same table will be lost anyway.

Thurallor's solution is certainly interesting, it basically does the same as Turbine's own I/O method by converting it to Lua-code. Only problem is that you need to wrap the result in a single huge string which makes pretty unreadable files, and I'm not too sure that you won't eventually run into any line-length limitation of the interpreter or underlying file I/O.

I decided to mix the solutions, I just write out numbers as string and parse them back with loadstring(), and prefix the strings so know if it was was number or not:
Code:

function convSave(obj)
        if type(obj) == "number" then
                local text = "n:" .. tostring(obj);
                return string.gsub(text, ",", ".");
        elseif type(obj) == "string" then
                return "s:" .. obj;
        elseif type(obj) == "boolean" then
                return obj;
        elseif type(obj) == "table" then
                local newt = {};
                for k, v in pairs(obj) do
                        if type(k) ~= "number" and type(k) ~= "string" then
                                -- unsupported keys
                        else
                                newt[convSave(k)] = convSave(v);
                        end
                end
                return newt;
        end
end

function convLoad(obj)
        if type(obj) == "string" then
                obj_id = string.sub(obj, 1, 2);
                if obj_id == "n:" then
                        -- need to run it through interpreter, since tonumber() may only accept ","
                        local readnum = loadstring("return " .. string.sub(obj, 3));
                        return readnum and readnum();
                elseif obj_id == "s:" then
                        return string.sub(obj, 3)
                end
        elseif type(obj) == "boolean" then
                return obj;
        elseif type(obj) == "table" then
                local newt = {}
                for k, v in pairs(obj) do
                        local key = convLoad(k);
                        if key ~= nil then
                                newt[key] = convLoad(v);
                        end
                end
                return newt;
        end
end

So a table like
{ [5] = 1.23, ["foo"] = "bar"}
becomes
{ ["n:5"] = "n:1.23", ["s:foo"] = "s:bar" }
which can be converted back in all locales.

I could also check for current locale and use gsub again and use tonumber() instead of loadstring() for numbers, and I haven't investigated if Lotro can handle non-printable characters or if I should use string.format("%q", ...) too...


All times are GMT -5. The time now is 10:34 AM.

vBulletin® - Copyright ©2000 - 2019, Jelsoft Enterprises Ltd.
© MMOUI