Scraping a website with content added via javascript with Selenium in Python -

-

I am trying to scrape data from a website using selenium and phatomaj in Python. However, this website is adding the data I am interested in via javascript. Is there a way to wait for data before withdrawing selenium? So far, we have tried: 

  importExport import webdriver as selenium.webdriver UI as phonomicon.webdriver.support.ui phantomjs = '/ usr / local / bin / Phantomjs' url = '[Redacted]' with contextlib.closing (webdriver.PhantomJS (phantomjs)) as driver: driver.get (url) wait = ui.WebDriverWait (driver, 10) wait.until (lambda driver : Driver.execute_script ("return document" ("[Redacted] ')) driver.execute_script (" return document.getElementById (\ "myID \") .intermentById (\ "myID \") .interTextById (\ "myID \") .interText ")

Unfortunately, this code raises selenium.common.exceptions.TimeoutException: message: none why We are unable to change the contents of the id .

We are using PhantmJs 1.9.7, Python 2.7.5 and Virtual and Selenium 2.41.0. The correct way to do or we have forgotten something

edit

@ExperimentsWithCode after following the comment, we have tried to load the content To: looping: driver 

  with referencelib.closing (webdriver.PhantomJS (phantomjs)) as driver: driver.get (url) wait = ui.WebDriverWait (driver, 10) found = found Not gone: try : Wait.until (Lambda driver: driver.execute_script ("return document.getElementById (\" myID \ "). InnerText ") foundwith ('[redacted]')) driver.execute_script (" Return Document.getElementById (\ "myID \"). InnerText =) found = Also true: print "not found" near  < / Pre>


Comments

Post a Comment

Popular posts from this blog

import - Python ImportError: No module named wmi -

-
I followed the instructions for downloading WMI for Python When I try to run the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = wmi.WMI (): if Raw_input ("restart% s?"% S.Caption) .upper () == "Y": s.StartService () I got the error < Code> traceback (most recent call end): file ". \", Line 1, edit: I am using Python 2.7.6 EDIT2: I am running 64-bit Windows 2008 R2, and I've downloaded WMI-1.4.9.zip (md5). I removed the content and put it in D: Python \ Tools \ Scripts I executed dragon setup.py.install I said D : \ Python \ tools \ scripts to% PATH% and when I execute the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = Wmi.WMI (): If raw_input ("Restart% s?"% S. Caption). () == "Y": s.StartService () error me Gets traceback (most recent call final): File ". See \", line 1, &
Read more

Editing Python Class in Shell and SQLAlchemy -

-
I am working on the terminal on a shell script after this tutorial SQLAlchemy tutorial. I need to type gt; & Gt; & Gt; Sqlalchemy import column, integer, string & gt; & Gt; & Gt; Class user (base): __tablename__ = 'users' id = column (integer, primary_key = true) name = column (string) absolute name = column (string) password = column (string) def __repr __ (self): return "& lt ; User (name = '% s', absolute name = '% s', password = '% s') & gt; "% (Self.name, self.fullname, self.password) After the problem I have typed the password = column (string) I hit twice and changed .... >> I then took everything back but then an error was thrown because the class already exists ... I am not completely sure how to fix it. How do I open that slip in the shell script and edit it (Add DF to DRR) The error thrown below: / user / giripeters / TFSQLLX / Lib / python2.7 / site-packages / sqlalchemy / ext / decla
Read more

c# - MySQL Parameterized Select Query joining tables issue -

-
I am using parameterized select questions in conjunction with my program and it works entirely, When I try to join the table Everything I have done is a small fraction: using (MySqlCommand cmd = new MySqlCommand (paramQuery) ToSql (), Connection)) /etc/ paramQuery.ToSql () Equal: "Select TableOne ID, Tableon, Department, Tablet. One by name table, Table 2 h Yes TableOnEID = @Pharmak "* / for (Int Index = 0; Index and Lt; ParamariX Parameter (.c calculation; Index ++) Cmd.Parameters.AddWithValue (paramQuery.Parameters (). ElementAt (index). , ParamQuery.Parameters (). ElementAt (index) .Value); / * ParamQuery.Parameters (). ElementAt (index) .k = "@most 0" paramQuery.Parameters (). ElementAt (index) .Value = "tableTwo.ID" * / MySqlDataReader Reader = cmd.ExecuteReader (); While (reader.Read ()) {// do stuff} } One of the tables I attempt to join is For the table, two with the same id for all. Is there something that I'm doing wrong?
Read more