TalkPHP - Help on Custom Data Storage

TalkPHP (http://www.talkphp.com/forums.php)

- Advanced PHP Programming (http://www.talkphp.com/advanced-php-programming/)

- - Help on Custom Data Storage (http://www.talkphp.com/advanced-php-programming/4281-help-custom-data-storage.html)

Help on Custom Data Storage

I need a custom data storage function which stores data on a text file. I need to make it only update 1 line at a time because I am having heavy requests and it needs to make several edits in a second.

I know it will be easier using database, but I (insist that I) need to use custom data storage function. Can anyone help please?

If you INSIST on using a file based data storage system ( Which will grow to be a slow heavy fiend after a while ) use XML or if you want a purely text file type storage system ( waste of time ) than you can use serialize/unserialize and use line # as a type of ID

Just delve into the lovely filesystem functions. Without any specific issues, problems, questions, we can't really be of any use to you.

If we were to start providing help on this topic, it could end up being useless since we don't know what you want/require/need. So, give us some context and ask some specific questions. :-)

I would suggest not using XML, simply because it can be a pain to parse. Instead I would store the data in JSON format.

Although, it is best to use a SQL database. They are much faster and handle big databases quickly.

If you don't mind my asking, why can't you use a database? There is -no- advantage to using a text file if you are working with either high volume traffic or lots of data.

Past that we need to know what you are doing to advise you on a custom data format.

I would use an XML file and use SimpleXML to retrieve the data, it's extremely easy. I store my page cache in XML files and its as easy as...

$xml->tag_name to grab the data. Ofcoarse, it can get a bit more complicated for nested items, but even that is pretty simple.

See :
http://us2.php.net/manual/en/functio...-load-file.php
http://us2.php.net/manual/en/functio...oad-string.php

XML for sure, there are some good parsers out there that will be able to index it fairly quickly. Also when using XML you will later be able to easily import it all into a database when your files get to large.

Quote:

Originally Posted by CoryMathews (Post 24115)

Also when using XML you will later be able to easily import it all into a database when your files get to large.

That's not really an issue is it? It is fairly trivial to import any data source "into a database"—there is nothing special about XML in that respect.

I still invite the OP to post a little more about what they want, why and any specific questions they have.

Ok, that's awful lots of replies at a time. I will answer the question 1 by 1:

@Enfernikus:
I'm already using serialize/unserialize, but the down side I see is that variables updated in a session gets overwritten by another session later on. Understand what I mean?

@Salathe:
I'm actually testing this er "method" or idea before implementing into an application. That application's requirements needs to stay basic which means the user only needs a web server and PHP to make it work. Therefore I'm trying my best to avoid using database.

@ETByrne:
Well, I really need a format that stores 1 row of array at 1 line, so that whenever I need to update them, I just update that particular line without touching the rest, unless PHP is unable to handle them...

@Village_Idiot:
Look at my answer to Salathe.
I'm only storing a few things, because it's about a script to download some files to the server, so it only needs to store the filename, the id, the download speed, currently downloaded file size and the total file size. I hope I'm clear enough for you guys to answer...

Seems like a lot of people recommend XML, I'll have a look at it first.

If it's still unclear to some of you what I actually want, here's a list of it
1. I only need to store a few strings and few numbers, but in rows, which means I will be retrieving it in 2 dimension arrays.
2. It's best if I can only update 1 line of the file and not touch the others.
3. The datas will be deleted after a period of time so the file will basically not grow too large, but the fact is that it will get updated very often (almost 1 time per 1.5s per user) which means I really need a way where I can update it without affecting other values.

Really long post but hopefully I'm clear enough.

It sounds like you need to look into SQLite or Sessions

I will actually like to know more about sessions. Is session being updated globally (I mean user's) when a script hasn't finish executing?

EDIT: I will prefer not to use SQLite because it requires the user to install an extension, I want to run it as native as possible.

Sessions persist until the user leaves the page ( I believe ) or until you destroy them

You might need to make it clearer, like I have a php script still running and it's updating the session, then if I initiate another script (like say the first script was index.php, then the second was get.php) will it get the updated session or the one before it?

Ok, I have tested the sessions method and proved that it doesn't work. I created 2 files as below:

Code:

<?php

session_start();



set_time_limit(0);

for (;;) {

        $i++;

        $_SESSION['test'] = $i;

}

?>

This is the code that modifies the session variable. Another file to retrieve it here:

Code:

<?php

session_start();

var_dump($_SESSION['test']);

?>

Assuming it should work, I ran the first file, then wait a few seconds, and tried to access the second file. But the second file just kept loading and seems like it will never stop. Seems like it means its not applicable in my circumstances.

Any more ideas other than XML ?

Quote:

EDIT: I will prefer not to use SQLite because it requires the user to install an extension, I want to run it as native as possible.

I was under the impression SQLite was enabled by default in php, ie.e it comes pre-bundled.

Hmm, I have actually no knowledge at all about SQLite. Does it need configuration? Does it come with every shared hosts?

Quote:

Does it need configuration?

Not really, just needs to be enabled (which mostly it is)

Quote:

Does it come with every shared hosts?

No idea, I wasn't under the impression this was to be a distributed script. However seen as the PHP manual says its enabled by default, I'm assuming it more than likely will be, I could be wrong ofc.

Sample sqlite script:

PHP Code:


		
			
try
{
    $db = new SQLiteDatabase('sqlite.db');
    $r = $db->arrayQuery('SELECT * from tbl WHERE col = 2');
    var_dump($r);
}
catch (SQLiteException $e)
{
    die($e->getMessage());
}

It can even do transactions:

PHP Code:


		
			
try
{
    $db = new SQLiteDatabase('sqlite.db');
    $db->query('BEGIN;
        CREATE TABLE test ( id UNSIGNED MEDIUMINT(8) PRIMARY KEY, data TEXT NOT NULL);
        INSERT INTO test (data) VALUES(\'Data\');
        INSERT INTO test (data) VALUES(\'Data2\');
        COMMIT;');

    echo $db->changes() . ' Rows inserted.'; //will echo '2 Rows inserted.
}
catch (SQLiteException $e)
{
    die($e->getMessage());
}

SQLite is available, and enabled by default, as of PHP 5. No doubt some hosting providers will disable it for some unfathomable reason but that should only be a small portion of hosts. For PHP 4, there is a PECL extension available.

Ok, SQLite seems nice, I will have a try. Just in case it doesn't work (in most cases, I think it doesn't) is there anymore alternatives? I'm afraid I'm gonna need to dry my brain cells trying to think up another way.

You could always go back to the beginning of the topic and create your own custom storage method.