I am not familiar with the editor myself, but every editor like TinyMCE (moxiecode) and WYSIWYG (What You See Is What You Get) have their own documentation. It should be discribed in fair detail how to modify the code to your likings. Otherwise, there'll be some good modification(sites) for your editor. Maybe it's smart to check those out.
I, and I might speak for more, can honestly say that without the system or experience with the editor it's fairly implausible/impossible that we can help you out. But as I see it as I get it, it has a small proportion of the SMARTY script for templating PHP files.
Then you've got an error in your form, the transform/filter function or the mySQL function.
Make sure you're using method="POST" at the form. Next, make sure the field has a name (name="field"). Then, when you POST it, you can retrieve it with $field = $_POST['field']; Next, you can apply all your filters and make sure that you transform the BBCode, else the function has to be applied to every post, when every visitor visits each time. (nice sentence! ) That should do the trick. Plus, enable error reporting (error_reporting( E_ALL ^ E_NOTICE);)