python 2.7 - How can I use word_tokenize in nltk and keep the spaces? -

-

takes the word_tokenize function in nltk, as far as I understand, a string sentence and return Represents a list of all your words: 

  & gt; & Gt; & Gt; Import from nltk word_tokenize, wordpunct_tokenize & gt; & Gt; & Gt; S = ("Good muffins cost $ 3.88 in New York, please buy me two of them. \ N \ nThank you. \ N" "> Gt; & Gt; & Gt; Word_tokenize ('good', 'muffin', 'cost', '$', '3.88', 'in', 'new', 'york', 'please', 'buy', 'm' It is important to keep a blank space for further calculations, but 'so I would like to return it to it like the  word_tokenize : 
 
  [' good ' '', '', '' '', '', '', '', '', '', '' '', '' '' '', '' '' '', '' '' '' ' 'York', '', 'please', '', 'buy', '', 'm', '', 'two', '', 'ki', '', 'them', '.', 'Thank you', '.']]  
 
 How do I change / change it Can you / tweak  word_tokenize ?

Step 1: Break by string and spacing

Step 2: word_tokenize 

 < Code> using each word (split according to location in step 1)>> S = "Good muffins cost $ 3.88 in New York, please let me know \ n" & gt; & Gt; & Gt; Ll = [[word_tokenize (w), ''] s split () for w & gt; & Gt; & Gt; List ('*', '', '', '', 'cost' '' '' '$' ',' 3.88 ',' '' '' '' '' '', '', 'York', '.', '', 'Please', '', 'buy', '', 'me' '']

Comments

Post a Comment

Popular posts from this blog

import - Python ImportError: No module named wmi -

-
I followed the instructions for downloading WMI for Python When I try to run the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = wmi.WMI (): if Raw_input ("restart% s?"% S.Caption) .upper () == "Y": s.StartService () I got the error < Code> traceback (most recent call end): file ". \", Line 1, edit: I am using Python 2.7.6 EDIT2: I am running 64-bit Windows 2008 R2, and I've downloaded WMI-1.4.9.zip (md5). I removed the content and put it in D: Python \ Tools \ Scripts I executed dragon setup.py.install I said D : \ Python \ tools \ scripts to% PATH% and when I execute the code for C.Win32_Service (StartMode = "Auto", state = "Stopped") import wmi c = Wmi.WMI (): If raw_input ("Restart% s?"% S. Caption). () == "Y": s.StartService () error me Gets traceback (most recent call final): File ". See \", line 1, &
Read more

Editing Python Class in Shell and SQLAlchemy -

-
I am working on the terminal on a shell script after this tutorial SQLAlchemy tutorial. I need to type gt; & Gt; & Gt; Sqlalchemy import column, integer, string & gt; & Gt; & Gt; Class user (base): __tablename__ = 'users' id = column (integer, primary_key = true) name = column (string) absolute name = column (string) password = column (string) def __repr __ (self): return "& lt ; User (name = '% s', absolute name = '% s', password = '% s') & gt; "% (Self.name, self.fullname, self.password) After the problem I have typed the password = column (string) I hit twice and changed .... >> I then took everything back but then an error was thrown because the class already exists ... I am not completely sure how to fix it. How do I open that slip in the shell script and edit it (Add DF to DRR) The error thrown below: / user / giripeters / TFSQLLX / Lib / python2.7 / site-packages / sqlalchemy / ext / decla
Read more

c# - MySQL Parameterized Select Query joining tables issue -

-
I am using parameterized select questions in conjunction with my program and it works entirely, When I try to join the table Everything I have done is a small fraction: using (MySqlCommand cmd = new MySqlCommand (paramQuery) ToSql (), Connection)) /etc/ paramQuery.ToSql () Equal: "Select TableOne ID, Tableon, Department, Tablet. One by name table, Table 2 h Yes TableOnEID = @Pharmak "* / for (Int Index = 0; Index and Lt; ParamariX Parameter (.c calculation; Index ++) Cmd.Parameters.AddWithValue (paramQuery.Parameters (). ElementAt (index). , ParamQuery.Parameters (). ElementAt (index) .Value); / * ParamQuery.Parameters (). ElementAt (index) .k = "@most 0" paramQuery.Parameters (). ElementAt (index) .Value = "tableTwo.ID" * / MySqlDataReader Reader = cmd.ExecuteReader (); While (reader.Read ()) {// do stuff} } One of the tables I attempt to join is For the table, two with the same id for all. Is there something that I'm doing wrong?
Read more