There have been a number of bugs in the underlying API that caused client crashes when unloading. Most of them have been fixed such as the Equipped Items bug and failure to remove registered commands.
I had posted a list of known issues and how to reproduce them in the Current API Issues & Discussion thread but I stopped working on it when Turbine stopped supporting Lua two and a half years ago. I don't recall if any of them were bugs specific to crashes on unload.
I don't use the options panel in the plugin manager as it is incredibly buggy and has been known to cause problems in the past. If you are using it, I would suggest disabling it and seeing if your crashes stop, and if so, simply provide your own window for your options panel. Otherwise, you will likely have to resort to disabling as much functionality as you can and reintroduce it a little at a time until you can isolate the source.
The biggest issue with multiple plugins in the same apartment has to do with stepping on each other's event handlers which doesn't generally cause crashes, just unintended functionality. If you have been loading just your one plugin and still getting client crashes on unload then it is not a problem of a conflict with another plugin sharing an apartment. Having too many private apartments can cause problems as each one eats up a lot of memory and LotRO is notorious for poor memory management so, if possible, don't use a private apartment.
EDIT: I was just browsing the Known Issues thread and saw that passing an invalid parameter to MenuItemList:Add() would destabilize the client and would cause a crash on unload, so there are still a few known issues that crash on exit. You might want to review the entire list to see if any of the issues seem similar to anything you are doing: