|
|
| Author |
Message |
|
| jwatte |
 Posted: Sat Oct 09, 2010 6:29 am |
 |
|
|
User
Joined: 10 Feb 2010
Posts: 34
|
Dunno how "advanced" this question is, but here goes:
I have an ETS table. It's used to serve web requests, as a cache for slow source data. It's about 50 MB in size. The table is protected and owned by a gen_server. Writes happen through handle_cast(), whereas reads happen in the process of whoever calls the read request (which generally is a mochiweb request handler -- but that's beside the point).
This table is populated by slowly rotating through source data that comes from slower media. There is also a timer which purges records that have not been updated after some time (to remove stale records). The source data is much more efficient to rotate through like this, than to search using random access, so I'd like to avoid a traditional memcached-like topology.
Once in a while (say, every 10 minutes), I want to dump this table to disk. This is so that, if the server has to be bounced, I can warm restart into a full (but outdated) set of data, instead of having to slowly re-populate the cache from the slow-cycling refresh source.
Questions:
1) I can see two ways of doing the dump and restore. Either I can create a dets table, call to_dets(), and then close the dets table. Or I can use tab2file() to dump the terms straight to file. In both cases, I will do an "atomic" save by dumping to a temp file, then using "rename()" to rename from temp file to real file name, clobbering any old file (this works on UNIX, not on Windows; on Windows there is a small window where the destination file won't exist).
Which if these two ways is likely to be more efficient? I am interested in at least two metrics:
- I/O throughput
- avoiding a long period of locking out other readers
2) The tab2file() documentation talks about public tables being edited while the file is created. Does this mean a protected table has to be dumped by the owning process? Ideally, I'd like to spawn a new process for the file dumping, to avoid locking out readers or writers. It seems file dumping only really need read/iteration access (a la safe_fixtable() and first()/next()).
Is it possible to call tab2file() on an ETS table from a non-owning process on a protected table?
Is it possible to call to_dets() on an ETS table from a non-owning process on a protected table? |
|
|
| Back to top |
 
|
|
|
All times are GMT
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
|
|
|
Digg It
Del.icio.us
Reddit
Facebook
Stumble Upon
Technorati
