Yield results asynchronously in python using multiprocessing or twisted -

-

I have an embarrassing parallel application, where the order of results does not matter.

I have a function and a list of 1000 arguments on which it works. 

  def _process_parallel (function, args_list, args_dict = {});   
 I have written a few multi codes for this parallelize. ..: num_tasks = lane (args_list) num_tasks_returned_ptr = [0] def _callback (result): num_tasks_returned_ptr [0] + = 1 # All jobs must be executed asynconously apply_results = [__POOL __ send args to apply_async (function, arg , Args_dict, _callback). Args_list] Wait for # until all tasks are processed num_tasks_returned_ptr [0]

I think the memory footprint is too high, the results of the function are not left at present until all the results are processed.

What I would like to do instead, there is something where the results are not stored after the execution. Something like this: 

  def _process_parallel (function, args_list, args_dict = {}): # Send all jobs to the result in somepackage.apply_async to be executed asynconously (function, arg, args_dict, _callback): yield result

I can not seem to find a way in multiplying I've heard good things about twisted, but I'm not sure this is simple After this for work

make a dragon generator You know the results regarding the area to calculate the asynchronous methods and they come like that causes them?


Comments

Post a Comment

Popular posts from this blog

import - Python ImportError: No module named wmi -

-
I followed the instructions for downloading WMI for Python When I try to run the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = wmi.WMI (): if Raw_input ("restart% s?"% S.Caption) .upper () == "Y": s.StartService () I got the error traceback (most recent call end): file ". \", Line 1, edit: I am using Python 2.7.6 EDIT2: I am running 64-bit Windows 2008 R2, and I've downloaded WMI-1.4.9.zip (md5). I removed the content and put it in D: Python \ Tools \ Scripts I executed dragon setup.py.install I said D : \ Python \ tools \ scripts to% PATH% and when I execute the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = Wmi.WMI (): If raw_input ("Restart% s?"% S. Caption). () == "Y": s.StartService () error me Gets traceback (most recent call final): File ". See \", line 1, & lt; Modu...
Read more

Editing Python Class in Shell and SQLAlchemy -

-
I am working on the terminal on a shell script after this tutorial SQLAlchemy tutorial. I need to type gt; & Gt; & Gt; Sqlalchemy import column, integer, string & gt; & Gt; & Gt; Class user (base): __tablename__ = 'users' id = column (integer, primary_key = true) name = column (string) absolute name = column (string) password = column (string) def __repr __ (self): return "& lt ; User (name = '% s', absolute name = '% s', password = '% s') & gt; "% (Self.name, self.fullname, self.password) After the problem I have typed the password = column (string) I hit twice and changed .... >> I then took everything back but then an error was thrown because the class already exists ... I am not completely sure how to fix it. How do I open that slip in the shell script and edit it (Add DF to DRR) The error thrown below: / user / giripeters / TFSQLLX / Lib / python2.7 / site-packages / sqlalchemy / ext / decla...
Read more

lua - HowTo create a fuel bar -

-
I would like to create a fuel bar (empty with full tank). When the user touches the screen, the fuel once Dairejh these is my code: lifeBar = display.newImage ( "fuel_bar1.png") lifeBar.anchorX = 0 lifeboats. ======================== Lifetime Y = 37 screens Group: The Insert function (called Laifbar) However, I fuel Kansmoshn (equivalent to "pressed" variable to reduce the fuel tank "true") Have used. When pressed == is true, it means that the user is touching the screen: processed (fuel) processed () if life & gt; 0 and pressed == true remains = life - life 1 Life = life =. Wide-1 remains Value.text = string.format ("% d", life) End of this This function is active as the entrance event: runtime: addEventListener ( "Antrfrem" fuel Snsadna) it is very well, but my question is: pressed variable each time 5 seconds (once = = True) How is it possible to reduce the vitality width? I tried to add it to my touch even...
Read more