Skip to main content
100%

NLP Concepts with spaCy. Code examples released under CC0 https://creativecommons.org/choose/zero/, other text released under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/

✓ Published0🌍 Public
Aabehmiel
Last edited Aug 11, 2017
Created on Nov 16, 2017

This notebook introduces core NLP concepts using the spaCy library, demonstrating how to analyze text structure through part-of-speech tagging and dependency parsing. It explains grammatical abstractions like sentences and word functions while acknowledging their limitations, particularly for poetry. The code example uses spaCy’s built-in pipeline to process English sentences, showing token attributes and linguistic annotations. Written in Python 2.7, it also mentions alternatives like NLTK and resources for other languages. The notebook’s content mixes explanatory prose with executable cells, offering a beginner-friendly tour of how language can be computationally decomposed.

AI-generated description

Similar vizzes

Loading thumbnail…

Spark Notes

This example shows a text-analytics pipeline that annotates a CSV-loaded corpus of academic articles. It renames raw columns to meaningful fields such as `article`, `journal`, `date`, and `text`, then uses Spark NLP’s `PretrainedPipeline("explain_document_ml")` to transform the data, adding tokens and other linguistic annotations. The code demonstrates schema inspection via `printSchema` and previews the resulting token column with `select("token").show()`, highlighting how Spark DataFrames integrate with a pretrained NLP model for scalable document processing.

CCliffordAnderson
63% match
Loading thumbnail…

Gist d2f2bc6be1716820160ed5183ea07eaa

This example shows a browser-based text analytics pipeline that scrapes and processes web content before visualizing topic models. The code first uses Selenium with Chrome WebDriver to scrape data, then applies NLP libraries including NLTK, spaCy, TextBlob, and gensim’s LdaModel for topic modeling and coherence scoring. The final visualization employs Plotly’s `plotly.express` and `plotly.graph_objs` to render interactive charts, with `pyo.init_notebook_mode()` enabling notebook display. The rendering approach is evident from the use of `go` traces and `px` figures, though the specific chart type is not shown in the code snippet. The data source is dynamic web pages, as indicated by the Selenium scraping logic.

AArizonaTay
62% match
Loading thumbnail…

Gist e049bb6d7696d1b2d4ff242e68fcd062

This example analyzes text data by performing topic modeling and visualization. It shows the resulting topics and their associated terms, revealing latent themes in the dataset. The code uses NLTK for tokenization and stopword removal, TextBlob for sentiment analysis, and Gensim’s LdaModel and CoherenceModel for topic extraction. Plotly’s `scatter` and `bar` traces render the interactive visualizations, with data processed from a pandas DataFrame. The rendering approach relies on Plotly’s offline mode, initiated via `pyo.init_notebook_mode()`.

AArizonaTay
60% match
Loading thumbnail…

Buzzfeed article scraper for NLP

This example scrapes a Buzzfeed article and converts its text into clean, tokenized sentences for natural language processing. It shows the pipeline from raw HTML to normalized, sentence-segmented data. The scraper uses BeautifulSoup and requests to extract paragraph elements with the class `subbuzz-text`. Text cleaning relies on the `unicodedata` module and regex via `re`, while sentence tokenization uses NLTK’s Punkt tokenizer. The final output is a single string of joined sentences, ready for further NLP analysis.

Aabehmiel
60% match
Loading thumbnail…

Gist 6499e03ffdfc7479408da70f20af3625

This visualization maps the semantic relationships between words extracted from a text corpus. It shows a network of nodes and edges, where connected words cluster together based on their contextual similarity. The layout uses a force-directed graph, likely powered by d3.forceSimulation, to position nodes and links. The code loads a spaCy NLP model (`en_core_web_lg`) to compute word vectors, and filters out stop words using NLTK’s stopwords list before processing the raw text.

AArizonaTay
58% match
Loading thumbnail…

Gist dec05c8e23491da95f80ba891c35100a

This example trains a custom named entity recognition (NER) model using spaCy to identify sponsor mentions, product names, and promo codes in social media captions. It first preprocesses a data frame of captions by stripping non-alphabetic characters, then initializes a pre-trained English model and adds custom labels. Training uses 10 epochs with minibatch compounding and dropout, iterating over hand-labeled examples that mark entity spans. After training, the model is saved to disk and reloaded to test inference on new text, demonstrating the pipeline’s ability to extract promotional entities from informal, hashtag-heavy captions.

AArizonaTay
57% match
Loading thumbnail…

Generative AI for News Media.ipynb

This notebook demonstrates how GPT-3 can be prompted for journalistic tasks like brainstorming article ideas and summarizing abstracts, while also outlining model limitations such as hallucination and bias. It uses the OpenAI API through Python code with predefined prompts, and includes a Colab badge link for interactive access. The content focuses on text generation examples rather than data visualization, relying on markdown explanations and code cells to illustrate prompting techniques.

NNickDiakopoulos
56% match
Loading thumbnail…

Gist 9176cfc4d4149c55845f

This IPython Notebook tutorial shows users how to get comfortable with markdown cells and basic Python coding tasks. It walks through editing text with markdown syntax and writing small code snippets, such as splitting a sentence into a list of words and using list splicing to extract the last few items. The notebook relies on the IPython Notebook API for rendering and executing cells, with all code examples written in standard Python.

MMartiHearst
56% match