Erlang Mailing Lists
Author Message

<  Erlang  ~  Is is possible to parallelize QLC
vsp
Posted: Tue Feb 22, 2011 4:06 pm Reply with quote
User Joined: 03 Mar 2010 Posts: 14
Hi All,

I have a 32 fragmented table holding 2.5 million records in total. All fields are stored as binary.

I have defined a function say
Code:

 Filterfun(F1, F2, F3, F4)->
          if %Some logic holds
               true;
          ->
               false
          end


Now i have defined a QLC as follows:

Code:

QH = qlc:q([{
X#library.F1,
X#library.F2,.....
X#library.F25
}
|| X<- mnesia:table(library),
 Filterfun(F1, F2, F3, F4, F5)
]),
%The way to spit QH to a output(in this case webmachine)


next(QC,Fmt,FmtEnd)->
  R=qlc:next_answers(QC,100),
  if=:=[]->qlc:delete_cursor(QC),          {list_to_binary(FmtEnd),done};
    true->
       RF=lists:concat([Fmt(tuple_to_list(E))||E<-R]),
         {list_to_binary(RF),fun()->next(QC,Fmt,FmtEnd) end}
  end. 




it takes 1.5 hours to process..which is very slow. I feel qlc:next_answers is slow, How can we make it parallel in a way that cursor is consistent in moving forward across processes ?

Thanks a lot
Ram
Back to top View user's profile Send private message
dongdongwu
Posted: Thu Sep 20, 2012 6:44 am Reply with quote
User Joined: 19 Sep 2012 Posts: 236
His good friend Diane said: "Christian Louboutin Men Shoes was a magician, he make shoes is immediately put his female people with legs and advantage. He understands women wanted to do and can make them into beautiful Cinderella." Madonna often in its concert wearing Christian louboutin high heels , and some famous superstar like Angelina jolie, mariah Carey, beyonce Knowles, the famous Japanese singer YaYouMei Hamasaki helps Christian Louboutin Men Shoes set up its powerful position. The youngest customers will count Tom cruise's daughter sully cruz. Louboutin made for only a pair of handmade Christian Louboutin high heel Shoes! Want to be more fashion? Put on Christian Louboutin Outlet !
Candy colors of the chalaza high-heeled shoes with lolita type allure, set full finely gem blue "neon shoes" is to need to use "sexy" to describe. Each pair are worth careful appreciation of lithe and graceful fairy ludaoli, what kind of most let you move?Christian Louboutin Men Shoes that one brush red is always cannot resist the temptation, Christian Louboutin men outlet continue to use the days of high 8cm above slender heel proclaim the sexy and luxuriant. The bowknot on black pointed high-heeled shoes with sharp rivet concomitant, wild python met enchanting color printing grain, It is that pairs of high-heeled shoes lets Carrie more feminine flavour. Like Christian Louboutin for men her word.
Back to top View user's profile Send private message

Display posts from previous:  

All times are GMT
View next topic
View previous topic
Page 1 of 1
Erlang Forum - Trap Exit Forum Index  ~  Erlang
This forum is locked: you cannot post, reply to, or edit topics.

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You cannot download files in this forum
Hosted by Erlang Solutions    |    Powered by phpBB and NoseBleed v1.14