Paginated Flat File DB

davidapple · 01-01-2010, 03:57 AM

I found this old article on this forum, about paginated txt file databases.

I've set up a site that pulls data from a single-dimensional txt file database, but at the moment it's not paginated.

The text file...

Code:

<!-- Comment1 --><div class="class"><a href="http://www.url.com/1" target="_blank"><img src="http://www.url.com/image1.jpg" border="0"></a></div>
<!-- Comment2 --><div class="class"><a href="http://www.url.com/2" target="_blank"><img src="http://www.url.com/image2.jpg" border="0"></a></div>
<!-- Comment3 --><div class="class"><a href="http://www.url.com/3" target="_blank"><img src="http://www.url.com/image3.jpg" border="0"></a></div>
<!-- Comment4 --><div class="class"><a href="http://www.url.com/4" target="_blank"><img src="http://www.url.com/image4.jpg" border="0"></a></div>
...

The PHP code...

PHP Code:


			
<?php
$fp = fopen('flat-file-data.txt','r');
if (!$fp) {echo 'Cant open file.'; exit;}

while (!feof($fp)) {
$line = fgets($fp, 2048);
list ($field1, $field2) = split ('\|', $line);

echo ''.$field1.'';
echo ''.$field2.'';

$fp++;
}
fclose($fp);

?>

I have set | as the deliminator but my txt files don't contain any deliminators, only line breaks.

Is there a way of using foreach() and explode() to paginate the data?

I tried doing this but it's not quite working. A few of the lines become truncated...

PHP Code:


			
<?php
$file_handle = fopen('flat-file-data.txt', 'r');
if (!$file_handle) {echo 'Cant open file.'; exit;}

while (!feof($file_handle)) {

$line_of_text = fgets($file_handle, 2048);
$field = explode('\|', $line_of_text);

echo $field[0]. ' ';

$file_handle++;

}
fclose($file_handle);

?>

Any ideas?

delayedinsanity · 01-01-2010, 05:05 AM

I'm not quite sure why you're attempting to explode on a delimiter that doesn't exist. That just puts the same string that's already held in $line_of_text into the 0 index of an array. You could omit that and gain the same result in less code;

php Code:

$line = fgets( $fh, 2048 );
echo $line;

// or even

echo fgets( $fh, 2048 );

What you may want to do is use file() as opposed to the method you've employed. It reads files into an array, which you can then work off of.

php Code:

header( "Content-type: text/plain" );


// FILE_IGNORE_NEW_LINES only removes \n, not \r for some reason...
$lines = file( __FILE__, FILE_IGNORE_NEW_LINES );

print_r( $lines );

foreach ( $lines as $line ) {

    // So you have to trim if you're on Windoze.
    $line = trim( $line );

    if ( empty( $line ) )
        continue;

    echo $line, PHP_EOL;

}

You can drop the above into its own file and run it to see how it works. Then modify and incorporate as you see fit. ;)

davidapple · 01-01-2010, 11:23 AM

Thanks for this

Yes, this method is a big improvement. A lot simpler. This code pulls all of the data from the txt file using foreach(). However, for some reason the data is preceded by...

PHP Code:


			
Array
(
    [0] =>

Also, is there a way of paginating this using something like...

PHP Code:


			
$linesperpage = 10;
/* This line makes it so you can set the page in the query string of the URL */
if(isset($_GET['start'])) $start = $_GET['start']; else $start = 0;

if($start > 0)
{
/* I'm kind of getting stuck at this part */
}

Is there a way of combining something like this with the first bit?

Salathe · 01-01-2010, 07:13 PM

I would like to introduce another approach to your problem, one which you're probably not aware of so would've have considered yet. To so do, you will have to think a little out of the box and if you're new to using classes/objects then things might seem a bit weird.

Another concept in there is filtering user input using the filter extension. It just makes the common task of filtering input a bit easier by allowing us to quickly validate the input type (in this case, integers only) and optionally limit the values to a specific range (1…1000) or specify a default value if none is present (1).

PHP Code:


			
// Open our flat file database for reading

$lines = new SplFileObject('flat-file-data.txt', 'r');



// Grab the user-requested page number from $_GET (with max/min/default limits)

$page    = filter_input(INPUT_GET, 'page', FILTER_VALIDATE_INT, array(

    'options' => array(

        'default'   => 1,

        'max_range' => 1000,

        'min_range' => 1

    )

));



// Ten lines per page

$perpage = 10;



// How many lines into the file should we start at

$offset  = ($page - 1) * $perpage;



// Limit the lines to the start point and number of lines per page

$page    = new LimitIterator($lines, $offset, $perpage);



// If this page does not exist (offset is too big), show first page instead

if (iterator_count($page) === 0) {

    $page = new LimitIterator($lines, 0, $perpage);

}



// Loop over this page of lines, displaying each one

foreach ($page as $line) {

    echo $line;

}

P.S. The LimitIterator isn't documented beyond the basic method and arguments lists. If you have any questions, do ask me... they might be useful for when I get around to writing the documentation for that class.

davidapple · 01-02-2010, 05:47 PM

Thanks for this. This looks like exactly what I'm looking for. I've copied this code and tested it on my site but it's not working. It's just not pulling anything through to the site.

Any ideas why this could be? I had a look for missing quotes and stuff but it appear to be fine. Humm.. it's a mistery

davidapple · 01-02-2010, 06:25 PM

I have tried using a query string too but it's just not working. For example:

test.html?page=1

Salathe · 01-02-2010, 07:50 PM

Do you have error reporting/displaying set sufficiently high to see errors/warnings/notices?

At the top of your script try setting:

PHP Code:


			
ini_set('display_errors', 'On');
error_reporting(E_ALL);

Enfernikus · 01-02-2010, 10:30 PM

Not to hijack the thread but Salathe is there an approximation of when every object will be properly documented? The SPL is extremely useful and I make use of atleast one or two objects on every project now.

Salathe · 01-02-2010, 11:38 PM

Enfernikus, see your PM inbox.

davidapple · 01-03-2010, 02:18 AM

I added the error lines and this is the error message..

Fatal error: Call to undefined function filter_input() in /vhost/*thesitepath*/www/test.html on line 12

Any ideas?

(I have set my website htaccess file to allow php from within html files and it works perfectly on all of the other pages so this is not the problem.)

delayedinsanity · 01-03-2010, 02:27 AM

That's funny, because as soon as Salathe posted that I was going to throw in a mention that the filter_* functions weren't added until 5.2.0. I've ran into this problem twice in the past little while with a few public domain projects I've released. First time was using htmlentities double_encode argument, and the second was with filter_var.

Upgrade PHP and you'll be good to go. `aptitude update | aptitude safe-upgrade` and if you're not on Debian, upgrade to that while you're at it. :D

davidapple · 01-03-2010, 11:34 AM

I'm using a shared hosting program (Pick N Mix, Easyspace.com) so I'm not in control of which version of PHP I am using.

Bit of a shame because the filter function looks really useful. Any easy ways of getting around this?

Parvus · 01-03-2010, 12:10 PM

Seeing they use PHP 5.1.6 (after a long long page load o_O)
You could ask them if they could upgrade to 5.2 (since changing your host isn't a option I guess =P) or you will have to make your own function for the filter_input() function.

davidapple · 01-03-2010, 12:29 PM

A ha, thanks for letting em know about this. I'll give them a call tomorrow. If they can't upgrade, is there a way to achieve pagination with my txt file using PHP 5.1.6?

Salathe · 01-03-2010, 03:06 PM

The use of the filter extension bears little relation to the chunk of code which paginates the file's contents; it's just there as an easy way to get, filter, validate the provided page number.

You can always fall back on the old style:

PHP Code:


			
$page = 1;

if (isset($_GET['page']) 

    && ctype_digit($_GET['page']) 

    && $_GET['page'] <= 1000 

    && $_GET['page'] >= 1

) {

    $page = (int) $_GET['page'];

}

davidapple · 01-03-2010, 04:26 PM

This is fantastic. I'm getting really close to finishing this project now. I have tested this with a test file that looks a bit like this..

PHP Code:


			
line one
line two
line three
line four
...

And it works a treat. I'll have to get my head around adding next page and previous page buttons (using query string) but I'm sure I can figure that out on my own.

I'm afraid, I do still have a problem though

. This code doesn't appear to be working with my real txt file. Maybe because it's a lot bigger, with much longer lines. Here is an example of thee lines in my txt file...

PHP Code:


			
<!-- Comment Comment Comment Comment Comment Comment --><div class="thumb"><a href="http://verylongwesiteaddressblahblahblah.com/folderwithaverylongname/htmlfile.html?querystring=lalalalalala" target="_blank"><img src="http://www.url.com/thumbs/image.jpg" border="0"></a></div>
<!-- Comment Comment Comment Comment Comment Comment --><div class="thumb"><a href="http://verylongwesiteaddressblahblahblah.com/folderwithaverylongname/htmlfile.html?querystring=lalalalalala" target="_blank"><img src="http://www.url.com/thumbs/image.jpg" border="0"></a></div>
<!-- Comment Comment Comment Comment Comment Comment --><div class="thumb"><a href="http://verylongwesiteaddressblahblahblah.com/folderwithaverylongname/htmlfile.html?querystring=lalalalalala" target="_blank"><img src="http://www.url.com/thumbs/image.jpg" border="0"></a></div>

I am certain that the quotes are not causing any problems, it's just that all of the lines are always returned regardless of the page number.

(Also Salathe, what does the number 1000 do in the previous reply?)

Parvus · 01-04-2010, 05:14 AM

I'm guessing the 1000 in Salathe's post means as an example for the maximum of what your page number could be.
*$_GET['page'] <= max page number and $_GET['page'] >= min page number*

About all the lines always getting returned:
Are you sure your text file contains multiple lines and not just 1 really big one ?
Because if it is just 1 long line, the code would set the page to 1 and always get the 1 line.

davidapple · 01-08-2010, 10:46 PM

{Sorry I haven't updated this since last weekend, I can't use the internet on weekdays}

Ah right, so that's what the 1000 means.

I have checked and I am sure that I do definitely have line breaks at the end of the lines. I have about 350 line approximately but I need a system that will allow for the txt file to be potentially thousands of lines long! Is this possible?

I know that users have to load the whole txt file every time they view a page don't they? I think this is still going to be quicker than loading many images though.

Any more feedback on this issue would be greatly appreciated.

davidapple · 01-09-2010, 04:39 PM

I've had a play around with this again today but I'm still really struggling. I would love to crack this soon. Any more advice anyone?

Salathe · 01-09-2010, 05:42 PM

Without being able to work with the same input (the file) as you, any further help would be guess-work.