Romeo: Thou jests with NLP? Juliet: ‘tis true! (7.29.20 Class Exercise)

Using NLP to build a sarcasm classifier:

• Natural language processing (NLP) describes the process of training a model that can interpret human speech. Some of the capabilities of NLP include language translation, text generation, and more. This week, we built an RNN that can detect sarcasm in news headlines. We achieved this by tokenizing words in a dataset of news headlines labeled either sarcastic (1) or not sarcastic (0) to turn sentences into sequences of numbers, padded these sequences with zeros to make them all the same length, and built an RNN (uses an embedding layer). After fitting the model, I gave it a spin on the following examples (green check = correct prediction, red x = incorrect prediction).
  

✅[1.3710256e-08] = ~0 - “The child-care industry is on the brink of collapse. Congress must rescue it.” (Washington Post)
✅[2.8298592e-07] = ~0 - “Protests live updates: NYPD facing questions as demonstrations continue.” (Washington Post)
✅[3.2580751e-01] = ~0 - “US announces plans to remove 12,000 troops from Germany, with half coming home.” (Fox News)
✅[7.1725477e-03] = ~0 - “Justice Department to expand federal law enforcement presence in Midwestern cities.” (POLITICO)
✅[5.6891394e-09] = ~0 - “These Microbes May Have Survived 100 Million Years Beneath the Seafloor.” (The New York Times)
❌[1.5391873e-07] = ~0 - “Trump’s Summer of Love Is a Distant Memory Now.” (POLITICO)
✅[9.9453628e-01] = ~1 - “Michael Goodwin: Barr eats Nadler’s lunch during testimony.” (Fox News)
❌[1.6046785e-07] = ~0 - “A Republican Panic Button.” (The New York Times)
✅[8.8728958e-01] = ~1 - “Why You’ll See Two Figures for G.D.P. Decline: Very Big, and Huge.” (The New York Times)
❌[2.1694852e-05] = ~0 - “Joe Biden takes a non-virtual moment to muse on his basement, apocalyptic ads and being Trump’s ‘antithesis’.” (The Washington Post)

Not too bad! The training and testing scores showed that the model became overfit so these mixed results were expected (likely due to the very large vocabulary). Particularly model had a had time identifying sarcastic headlines but the cases that were missed are not super obvious; if I didn’t know the contexts, I would probably label them as not sarcastic too (I’m actually questioning if the ones it counted as not sarcastic really are, I’m really bad at picking up on sarcasm).

Text generation with an RNN:

• This week, we also used an RNN to generate text based on a learned piece of litterature. The model “learns” the text by tokenizing words to make a dictionary. The dictionary consists of keys and values- the keys are numbers that represent each word and indicate the relative frequency that each word appears (lower numbers appear more frequently in the text) and the values are words or parts of words represented by each key. The model trains by utilizing a function called ‘embedding’ as one of the model layers. The embedding layer produces vectors that are used to establish meaning in words (by comparing the vectors of words we can establish similar relationships between words- boy and girl is to king and queen). Below are two outputs of the model- we get a different output each time. Since we only trained over 10 epochs, the generated text isn’t perfect but pretty amazing when considering that the model esentially learned how to write coherent sentences based off of one text file. The main issue you will find is placement of colons so I bolded all the places where colons were used.

Round 1:

ROMEO: he had Slatue upon the fine of the town? KING EDWARD IV: Well, villain,–For no pluck HORSET: Most gracious knigeth hath might repuse his rage So much greater hour tideness me? Paging: My lord, she’s not come to my mistress. Thou letters he not, too much? He apry to look upon thy hand than hated? Thought will repair at me, resembly nor that ended this: Some that at the tomach lies that bred that ends On years, methoughts to maintain my schooly sprending tale, And all hever ribn’d ad in the trumpets, This world good namen o’ the first, to make Our city canst not right; Henry, all cold, kinsman. Feaths, take it you stuck That hath given him the place against thy prince, Hast thine boing with thinesell before thee. Jost? Durse: The cause of fraificiness that seest to see, And his confell’d is! Here did we percution their heart that offended voice Wor brows not. First Stander. Lords, twice or a poor soul. You beheld, it were as help to give: what company, now I tell you kiss, consunti

Round 2:

ROMEO: I would I hope his word: ‘tis already: Those wits, laid Beneminius; Left behind. DUCHESS OF YORK: O making, very, taility, for my butt, consent, As seeketh good as free frown, tormentaclion the fine marriage From thy dear to strike upon’t a sign the heart’s give was night, And we will be dlad rue by banishment, That they no carck of tast it offended or thy life, And, but your good due news cry ‘trumzeth, ladies flies begins to me: But Officer of the murdeit profort up: And I hope by rather, and you sit Then are crALINGBROKE: That’s my barrities, for thee. Farewell, you’re, unwill I give; for him Do you before I cannot that. LEONTES: They not round be raged. FLORIZEL: Here I be called my pare. If you call conspiracle my mast before The best is worse and time from on. TRANIO: A place; Which Hapting pendly-born: Duke, fair! O, the nobles home, cries Juliet. Second Servingman: I have not toe touch their header than he was busts A suffortune ears or peeting pirtued words; And would you

Neural machine translation with attention:

• Lastly, we built a Spanish to English translator. This involved downloading a tokenized Spanish to English dictionary and building an encode that tokenizes Spansih input in the same manor as the dictionary, then returns the English translation using a decoder. Below are some predictions made using the translator and the associated plots of attention weights - it did very well!
  

Spanish: hace mucho frio aqui.
Predicted translation: it s very cold here.

Spanish: esta es mi vida.
Predicted translation: this is my life.

Spanish: ¿todavia estan en casa?
Predicted translation: are you still at home?

Spanish: trata de averiguarlo.
Predicted translation: try to figure it out.