Jump to content

Efficient array storage for php


btherl

Recommended Posts

PHP's arrays have high overhead, particularly for complex structures such as

[code]$a[0] = array(
  'host' => 'www.phpfreaks.com',
  'path' => 'forums',
  'file' => 'index.php',
  'args' => 'action=post',
  'count' => 5,
  'potatoes' = true,
  ... etc etc
);[/code]

Are there any extensions for php which allow for more efficient data storage, at the expense of reduced flexibility?  I am looking for something similar to a C structure, where data is tightly packed, and names for the elements do not need to be stored.

The structure needs to support:

[list]
[*]Inserting at the end of the array
[*]Fetching data from any location in the array (indexed by integers, like a C array)
[*]Sorting (this could be complex)
[/list]

My main concern is that the labels for each data item (such as 'host', 'path') are repeates for EVERY element in my array $a.  This is a painful waste of space.  My secondary concern is that I do not want to use an entire zval to store a simple boolean, or a simple integer.  I would like to pack these values more tightly, even if accessing them becomes more costly.

Thanks for any advice :)
Link to comment
Share on other sites

[quote]I am looking for something similar to a C structure, where data is tightly packed, and names for the elements do not need to be stored.
[/quote]
Well, you don't HAVE to store the names of the elements

I think the default structure of an array is sorted by index
unless one happens to have a stringed key.


I don't know if I'm on the right track exactly to what you're asking but....
if you don't want an associative array, one with string indeces
and want to avoid the 'repeats'

maybe have a shema element at the beginning of the array to
aesthetically please you're coding....or whatever

but overall...you don't need the keys
and as for your question about adding to the end of the array
it should still work
Link to comment
Share on other sites

Thanks for the comments!

The reason it's a concern is that I want to process large amounts of data, and I want to sort the entire data set by various keys.  The larger data sets are 250k rows, and possibly larger.  The data would fit in memory, but the overhead of storing the data is currently much larger than the data itself.

Jenk, I'm not sure what you mean by "PHP uses C directly for primitives".  As I understand, php uses hash tables for arrays.

Zanus, there looks to be a small memory benefit to using integers instead of strings, based on small tests.  I will try it and see if it measurably improves memory usage, thanks :)

I don't think the underlying structure is different for numerically indexed arrays however.  It would be nice if it was :)  But memory usage is the same for numerically indexed arrays and arrays indexed by short strings.  I would expect lower memory usage if a simple array was being used.
Link to comment
Share on other sites

This thread is more than a year old. Please don't revive it unless you have something important to add.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.