Episode #354

Vapor Demo: Tokenizr

Series: Server-side Swift with Vapor

21 minutes
Published on September 7, 2018

This video is only available to subscribers. Get access to this video and 578 others.

Let's take what we have learned and build a simple web app. We'll leverage NSLinguisticTagger on the server and built a small UI that extracts names from provided text. We'll lean on everything we have used so far in this series: routes, templates, master templates, context data, and a little CSS to make the UI look nice.

Starting out

We'll start with the web template option when creating our project:

$ vapor new tokenizr --template=web

Then we'll build the project and generate an Xcode project.

$ vapor build
$ vapor xcode

Then I want to remove things we aren't going to use, so I'll delete all of the routes, the css file, and the image that is included with the template.

Creating the initial route

We can create our first route that renders a template:

router.get { req in -> Future<View> 
    return try req.view().render("home")

Now we need to create our template.

Creating the home template

We'll start with just some basic HTML so we can see it working.

  <h2 class="ui teal image header">Tokenizr</h2>
  <p>A tool to explore <code>NSLinguisticTagger</code>'s behavior.</p>

We can Build & Run in Xcode and visit the page to see this in action.

Next let's add our form to accept some text input:

  <form class="ui large form" action="/tokenize" method="post">
    <div class="field">
      <div class="ui left icon input">
        <textarea name="text" placeholder="Enter some text" rows="4" cols="40">#(text)</textarea>
        Example: <em>The American Red Cross was established in Washington, D.C., by Clara Barton.</em>
    <button class="ui fluid large teal submit button" type="submit">

Note that we're using some CSS classes from Semantic UI, which we will include next.

Embedding in a master template

We can move our overall HTML structure as well as common styles, scripts, and layout in a master template and use it from multiple templates.

Our master template looks like this:

<!doctype html>
<html lang="">

<meta charset="utf-8">
<meta http-equiv="x-ua-compatible" content="ie=edge">
  Tokenizr - #get(title)
<meta name="description" content="A site to browse NSLinguisticTagger's behavior.">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/semantic-ui@2.3.3/dist/semantic.min.css">
<script src="https://cdn.jsdelivr.net/npm/semantic-ui@2.3.3/dist/semantic.min.js"></script>

<link rel="stylesheet" href="/styles/main.css">



    <div class="ui middle aligned center aligned grid">
      <div class="column">

The styles that are referenced here can be found the source code for this episode.

Note that this has two dymnamic sections, #get(title) and #get(content). We need to provide these two variables in our home template:

#set("title") {

#set("content") {
  <h2 class="ui teal image header">Tokenizr</h2>
  <p>A tool to explore <code>NSLinguisticTagger</code>'s behavior.</p>



Now we can refresh and it will render these dynamic sections into the placeholders defined in the master template.

Handling the form post

Let's create another route to handle this form post:

 router.post("tokenize") { req -> Future<View> in
        let text: String = try req.content.syncGet(at: "text")

        let tagger = NSLinguisticTagger(tagSchemes: [.nameType], options: 0)
        tagger.string = text
        let range = NSRange(location: 0, length: text.utf16.count)
        let options: NSLinguisticTagger.Options = [.omitWhitespace, .omitPunctuation]
        let tags: [NSLinguisticTag] = [.personalName, .placeName, .organizationName]

        var matches: [String] = []
        tagger.enumerateTags(in: range, scheme: .nameType, options: options) { tag, tokenRange, _, _ in

            if let tag = tag, tags.contains(tag) {
                let tagString = (text as NSString).substring(with: tokenRange)

        let context = TokenizeResponse(text: text, matches: matches)

        return try req.view().render("home", context)

Note that we're creating a context object called TokenizeResponse to hand data over to the view, which is defined like this:

struct TokenizeResponse : Encodable {
    var text: String
    var matches: [String]

Displaying the results in the view

We'll move most of the HTML for displaying results into a partial template that we'll call _results.leaf. We can embed it like this:

  #if(matches) {

The results partial looks like this:

<div class="matches">
    #for(match in matches) {

And that's it! Re-run Xcode and see your completed demo!

This episode uses Leaf 3.0.1, Swift 4.2, Atom 1.30.0, Xcode 10.0-beta6, Vapor 3.0.8.