TalkPHP
 
 
Account Login
Latest Articles
» The basic usage of PHPTAL, a XML/XHTML template library for PHP
» Vulnerable methods and the areas they are commonly trusted in.
» Simple way to protect a form from bot
» The Basics On: How Session Stealing Works
» How to keep your forms from double posting data
IRC Channel
IRC Speech Bubble Join the friendly bunch on IRC...
(#TalkPHP on Freenode)

...Also available via a web interface.

See this thread for information on the TalkPHP Free Hugs Initiative™. Subject to availability.
Associates
Associates
CSS Tutorials
Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old 01-30-2013, 04:50 PM   #1 (permalink)
The Visitor
Newcomer 
 
Join Date: Jan 2013
Posts: 1
Thanks: 0
Vali is on a distinguished road
Default Efficient way to passing large objects between scripts?

Hi

I have a system set up a bit like a tree, where the trunk is the start and end point.
Ex:
Request goes to the controller (1)
That controller starts up multiple sub-controllers (N)
Each sub-controller starts some workers (n)
The workers return the data to the sub-controller (which does it's magic), which in turn returns the data to the main controller, so it can do it's magic before returning it to the user.

These scripts are spread between multiple servers (on the same GB network), usually there about 900 scripts started for every request, and the data passed between scripts is usually under 1MB (multi-dimensional arrays making up objects )

Right now, the way I pass the data is by json_encode in the worker and json_decode in the parent.
But, this is #1 to slow (and about 5x faster than serialize) and #2 takes WAY to much RAM (sometimes for 500KB of values it takes 60mb of ram, and this is per worker/child).

From one request that takes about 20sec, 10 to 15sec is usually only this json_encode/json_decode part.

So the question is:
- Is there a better way to transfer this data from one script to another (I need to use all the data in each script, so can't pass the ID and select from the global cache/db)

Please reply.
Vali is offline  
Reply With Quote
Reply



Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


All times are GMT. The time now is 12:22 AM.

 
     

Powered by vBulletin® Version 3.6.8
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.1.0
Inactive Reminders By Icora Web Design