Parsing numeric data from text file using Python -

-

I am trying to create a database from a numerical model output text file. There are four (4) rows of title block data in the text file, which are in accordance with the blocks of data in several rows (41,149), which are different from each 'internal' word and below it are some numerical data forms Line 1: Title Block Line 2: Title Block Line 3: Title Block Line 4: Title Block Line 5: Internal 1.0 (10 e 16.9) -1 Line 6: Data Data Data Data Line7: Data Data Data Data Line8 to Line8: Data Data Data Data Line26: Data Data Data Data Line27: Internal 1.0 (10 e 16.9) -1 Line 28: Data Data Data Data .. By all means 41,149 lines up to < P> Data blocks are not consistent size (i.e., there are more rows of data than some others). Thanks to a lot of help from this site, I am able to take 41,149 rows of data and organize each data block in different lists, from which I can parse and build a database. My problem is that this operation seems to last a long time. I was hoping someone could see the code given below and suggest me how I could be able to run it more efficiently. I can attach the model output file if necessary thanks! 

  inFile = 'CONFINED_AQIFER.DIS' string = ['internal'] rowList = [] # create a list of each line number where a data block starts with the open (infile) such as MyFile: For Num, the line in Enumerate (myFile, 1): if any (in the line for the string for S): rowList.append (num) # line number received from DF Milllino (filename, Leno) Function to: if lineno & lt; 1: If there are no lines 1: line = f.readlines (100000) type line ("first line line 1"), then f = open (filename) line_read = 0 is not a return: returns none if line_dill + Lane (line) & gt; = Lineno: return line [lineno-lines_read-1] lines_read + = len (rows) # Arrange each data block in a unique list and add the last list (fList) to the line in the fList = [] row (line (line List [1:])): combined list = [] I = row list [row] data = [] while i

< P> Some comments:

In Python 2, the xrange is always better than the range, the range creates the full list, while the exchange only gives an Iterator

See if you have these techniques Up to extrapolate more code.

Normally you do not want to allocate memory inside the loop (creating new lists)


Comments

Post a Comment

Popular posts from this blog

import - Python ImportError: No module named wmi -

-
I followed the instructions for downloading WMI for Python When I try to run the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = wmi.WMI (): if Raw_input ("restart% s?"% S.Caption) .upper () == "Y": s.StartService () I got the error < Code> traceback (most recent call end): file ". \", Line 1, edit: I am using Python 2.7.6 EDIT2: I am running 64-bit Windows 2008 R2, and I've downloaded WMI-1.4.9.zip (md5). I removed the content and put it in D: Python \ Tools \ Scripts I executed dragon setup.py.install I said D : \ Python \ tools \ scripts to% PATH% and when I execute the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = Wmi.WMI (): If raw_input ("Restart% s?"% S. Caption). () == "Y": s.StartService () error me Gets traceback (most recent call final): File ". See \", line 1, &
Read more

Editing Python Class in Shell and SQLAlchemy -

-
I am working on the terminal on a shell script after this tutorial SQLAlchemy tutorial. I need to type gt; & Gt; & Gt; Sqlalchemy import column, integer, string & gt; & Gt; & Gt; Class user (base): __tablename__ = 'users' id = column (integer, primary_key = true) name = column (string) absolute name = column (string) password = column (string) def __repr __ (self): return "& lt ; User (name = '% s', absolute name = '% s', password = '% s') & gt; "% (Self.name, self.fullname, self.password) After the problem I have typed the password = column (string) I hit twice and changed .... >> I then took everything back but then an error was thrown because the class already exists ... I am not completely sure how to fix it. How do I open that slip in the shell script and edit it (Add DF to DRR) The error thrown below: / user / giripeters / TFSQLLX / Lib / python2.7 / site-packages / sqlalchemy / ext / decla
Read more

c# - MySQL Parameterized Select Query joining tables issue -

-
I am using parameterized select questions in conjunction with my program and it works entirely, When I try to join the table Everything I have done is a small fraction: using (MySqlCommand cmd = new MySqlCommand (paramQuery) ToSql (), Connection)) /etc/ paramQuery.ToSql () Equal: "Select TableOne ID, Tableon, Department, Tablet. One by name table, Table 2 h Yes TableOnEID = @Pharmak "* / for (Int Index = 0; Index and Lt; ParamariX Parameter (.c calculation; Index ++) Cmd.Parameters.AddWithValue (paramQuery.Parameters (). ElementAt (index). , ParamQuery.Parameters (). ElementAt (index) .Value); / * ParamQuery.Parameters (). ElementAt (index) .k = "@most 0" paramQuery.Parameters (). ElementAt (index) .Value = "tableTwo.ID" * / MySqlDataReader Reader = cmd.ExecuteReader (); While (reader.Read ()) {// do stuff} } One of the tables I attempt to join is For the table, two with the same id for all. Is there something that I'm doing wrong?
Read more