LoTROInterface - Plugin support for German and French clients

LoTROInterface (https://www.lotrointerface.com/forums/index.php)

- Lua Programming Help (L) (https://www.lotrointerface.com/forums/forumdisplay.php?f=50)

- - Plugin support for German and French clients (https://www.lotrointerface.com/forums/showthread.php?t=1195)

Plugin support for German and French clients

Since there's not yet discussion about this, I thought I'd start one.

So what we have is a nasty bug with how data is serialized when using the internal Turbine.PluginData.Save method. What happens is that with German (and French?) clients the default decimal separator, which is "." (dot), gets converted to "," (comma). This can obviously break the save file badly since it's parsed as a Lua file. We can expect that this will be fixed by Turbine in the future, but plugin authors can react to this quite a bit faster. So, what should we do about it?

1) Each and every plugin author should find their own way to adjust plugins so that they work for all client versions. That is, if they want or bother to (it can be a lot of work).
2) We expect that those using other clients than English will start to use tools such as the fix provided by Vindar.

Personally I'd like to modify mine so that they do work for everyone. But then again does that interfere with the fix? I've also found it problematic to test this since it doesn't appear that anything short of installing a German client is enough to make this bug appear.

-Chiran

If you alias the Turbine.PluginData.Load and Turbine.PluginData.Save methods it will cause the fix to not change anything in your plugin like this:

Code:

local load_alias = Turbine.PluginData.Load;

local save_alias = Turbine.PluginData.Save;



-- now just use the alias functions to do the saving or loading

save_alias(Turbine.DataScope.Character, "PluginName", object_to_save);

local loaded_object = load_alias(Turbine.DataScope.Character, "PluginName");

Ah, that's a good tip, thanks!

-Chiran

Since Palantir has a hefty number of settings, I was planning on creating a couple of functions to iterate through the tables before saving, and after loading to convert numbers/bool to strings and back.

Quote:

Originally Posted by Digital_Utopia (Post 5431)

That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.

Quote:

Originally Posted by NuclearTonic (Post 5434)

That's exactly what the v1.5.2 version of TonicBars does. Copied strait after deepcopy provided by Turbine.

It might just be because I don't have the TurbinePlugins folder anymore, but I couldn't find that particular code. Instead, I just went with these functions:

Code:

function settingsEncode(t)

        local t2={};

        for k,v in pairs(t) do

                if(type(v)=="number")then

                        v=tostring(v);

                elseif (type(v)=="table")then

                        v=settingsEncode(v);

                end

                t2[k]=v;

        end

        return t2

end

Code:

function settingsDecode(t)

        local t2={};

        for k,v in pairs(t) do

                if(tonumber(v)~=nil)then

                        v=tonumber(v);

                elseif (type(v)=="table")then

                        v=settingsDecode(v);

                end

                t2[k]=v;

        end

        return t2

end

I implemented a slightly different solution for my plugin.

Basically, before I call Turbine.PluginData.Save(), I serialize my settings table (and subtables) into a string. The string is simply a snippet of Lua code that, when executed, returns my settings table. Then I save that single string with Turbine.PluginData.Save().

After I get the string back from Turbine.PluginData.Load(), I use the built-in loadstring() function to parse it back into a table.

Vindar's fix didn't work for me, because my settings table contains tables that look like this:

Code:

t = { "a", "b", "c" }

and Vindar's loader was returning them like this:

Code:

t = {

  ["1"] = "a",

  ["2"] = "b",

  ["3"] = "c"

}

Unfortunately, there is a difference. :) I hacked Vindar's code for a while, but I couldn't get it to work. So I implemented my alternative. Here's the source code:

Code:

function SaveSettings(settings)

    -- Serialize (workaround for Turbine localization bug)

    local saveStr = "return " .. Serialize(settings);

    Turbine.PluginData.Save(Turbine.DataScope.Character, "Stuff", saveStr, function()

        -- Finished saving

    end);

end



function LoadSettings()

    Turbine.PluginData.Load(Turbine.DataScope.Character, "Stuff", function(saveStr)

        if (saveStr) then

            -- Unserialize (workaround for Turbine localization bug)

            settings = assert(loadstring(saveStr))();

            if (not settings) then

                Turbine.Shell.WriteLine("Failed to parse Stuff.plugindata!");

            end

        end

    end);

end



function Serialize(obj)

    if (type(obj) == "nil") then

        return "nil";

    elseif (type(obj) == "boolean") then

        if (obj) then

            return "true";

        else

            return "false";

        end

    elseif (type(obj) == "number") then

        local text = tostring(obj);

        -- Change floating-point numbers to English format

        return string.gsub(text, ",", ".");

    elseif (type(obj) == "string") then

        return string.format("%q", obj);

    elseif (type(obj) == "table") then

        local text = "{";

        for i, v in pairs(obj) do

            local index = Serialize(i);

            local value = Serialize(v);

            if (value ~= nil) then

                local item = "[" .. index .. "]=" .. value .. ",";

                text = text .. item;

            end

        end

        text = string.gsub(text, ",$", "");

        text = text .. "}";

        return text;

    else

        return nil;

    end

end

You may notice the only escaping I'm doing for strings is replacing " with \" and \ with \\. I'm not sure if there's anything else I need to do, such as for strange foreign characters. Can anyone advise me?

Quote:

Originally Posted by Thurallor (Post 8635)

'Foreign' characters are encoded into UTF8. While the lua interpreter does not handle that encoding, lua strings are nothing more than arbitrary sequences of bytes, and they are transmitted to and fro the game engine and file system without change, so you generally don't need to worry about it for storing/retrieving strings.

Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.

Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"

Quote:

Originally Posted by Equendil (Post 8642)

Hi Equendil, thanks for the reply. I use your table browser plugin quite a lot. :)

Quote:

Full-proof code would probably require escaping line feeds and maybe other control characters though, as string literals within double quotes are supposed to be on one line.

Hmm, good point. And I do have some strings containing linefeeds. May explain some flaky behavior I've been seeing.

Edit: Maybe I can avoid the whole problem by using "long-bracket" literal strings: [[ ... ]].

Edit: Or by using string.format("%q", ...). Updated the source code.

Quote:

Edit: I also don't understand the purpose of "text = string.gsub(text, ",$", "");"

That's just me being anal. It turns

t = { "a", "b", }

into

t = { "a", "b" }

even though the former is completely acceptable in Lua. For the sake of maximizing efficiency, perhaps I should remove it.

Seems like I won't get around this stuff either...

Thurallor, I can't reproduce the problem of Vindar's code returning number indices as string indices. Are you sure you converted the saved data back?
But in any case it only solves the locale problem partially, it will still save numbers as "1,234" and hence still fails converting back at loading when switching client language.
Also I'm not sure why he needs that metatable and other "black magic", I'm pretty sure references to the same table will be lost anyway.

Thurallor's solution is certainly interesting, it basically does the same as Turbine's own I/O method by converting it to Lua-code. Only problem is that you need to wrap the result in a single huge string which makes pretty unreadable files, and I'm not too sure that you won't eventually run into any line-length limitation of the interpreter or underlying file I/O.

I decided to mix the solutions, I just write out numbers as string and parse them back with loadstring(), and prefix the strings so know if it was was number or not:

Code:

function convSave(obj)

        if type(obj) == "number" then

                local text = "n:" .. tostring(obj);

                return string.gsub(text, ",", ".");

        elseif type(obj) == "string" then

                return "s:" .. obj;

        elseif type(obj) == "boolean" then

                return obj;

        elseif type(obj) == "table" then

                local newt = {};

                for k, v in pairs(obj) do

                        if type(k) ~= "number" and type(k) ~= "string" then

                                -- unsupported keys

                        else

                                newt[convSave(k)] = convSave(v);

                        end

                end

                return newt;

        end

end



function convLoad(obj)

        if type(obj) == "string" then

                obj_id = string.sub(obj, 1, 2);

                if obj_id == "n:" then

                        -- need to run it through interpreter, since tonumber() may only accept ","

                        local readnum = loadstring("return " .. string.sub(obj, 3));

                        return readnum and readnum();

                elseif obj_id == "s:" then

                        return string.sub(obj, 3)

                end

        elseif type(obj) == "boolean" then

                return obj;

        elseif type(obj) == "table" then

                local newt = {}

                for k, v in pairs(obj) do

                        local key = convLoad(k);

                        if key ~= nil then

                                newt[key] = convLoad(v);

                        end

                end

                return newt;

        end

end

So a table like
{ [5] = 1.23, ["foo"] = "bar"}
becomes
{ ["n:5"] = "n:1.23", ["s:foo"] = "s:bar" }
which can be converted back in all locales.

I could also check for current locale and use gsub again and use tonumber() instead of loadstring() for numbers, and I haven't investigated if Lotro can handle non-printable characters or if I should use string.format("%q", ...) too...