Impactful Products!


Using AI to read 50,000 emails a day

Date: June 29, 2020
Read time: 3 min

Coronavirus has cut a large and uneven swathe through the world’s businesses. While some fields — hospitality, transportation, restaurants — have been crushed, other fields — grocery delivery, e-commerce — have been crushed in a very different way: with too much demand from sheltering consumers. This is the story of how Tolstoy, a NLP company based in San Francisco, helped an online garden retailer handle a massive inflow of customer emails as they waged their way through the pandemic.

Pandemic-level demand
While some businesses have seen their demand dry up overnight, other businesses have experienced record inflow. One of the activities consumers stuck at home have turned to is gardening. Garden retailers both in the US and Europe have seen orders skyrocket hundreds, or thousands, in percent growth.

One of these garden retailers approached Tolstoy with a dilemma. The coronavirus spigot had sent them several times more orders than they had capacity to fulfill. Correspondingly, they had received over 50,000 customer emails since the start of March — most of them angry emails inquiring on order status.

The company usually routed customer inquires through Freshdesk, an email platform, that helped categorize inbound mail. But now they were receiving correspondence from many sources, including direct email, which had no categorization applied. They were confronted with the prospect of reading through more than 50,000 emails — with thousands added per week — just to triage high priority emails, such as cancellation requests.

They estimated that if they hired 4–5 staff, it would take about a month to read the emails — during which tens of thousands more emails would arrive.

AI to process emails
Tolstoy took a look at the company’s email data. Email text data can often be messy — everything from the sender fields to legal disclaimers are included, so we cleaned it up with some preprocessing. This meant stripping out all forwarded emails and auto-email generated text, after which we normalized the text by removing punctuation and capitalization.

Next, we considered the training set of about 3,000 categorized emails. These emails were classified through Freshdesk, which meant that customers picked the category. While most of them were useful categories, customers often picked wrong categories — for example, some selected “Cancel” when they actually wanted to inquire about a new product, or “Delivery query” when they meant to cancel. We manually corrected and labelled a set of more than 5,000 emails to assert “ground truth”.

We also balanced the limitations of the training data by hard-coding certain heuristics into the predictions.

After testing different models, we settled on a logistic regression, which is based on the frequency of words found in the email text. To help the model focus on the more important features, we only included words that surpassed a fixed number of appearances across the dataset and incorporated elastic net regularization.

Ultimately, we configured the model to read and categorize emails with ~98% accuracy, and 97–99% precision/recall (a measure of true/false positives captured), based on a set of 1,300 manually corrected emails. This was far more accurate than even the existing categorized emails via Freshdesk.

Record time
The garden retailer sent over their 50,000+ batch of emails on a Friday. We tuned the model on the following Monday, and returned the categorized results back on Tuesday.

This enabled the retailer to immediately start processing and responding to all their customer emails, which they had not been able to do for months. Finally, customers could get their answers and orders fulfilled or refunded.

From Tolstoy to the gardeners of the world, Here’s to hoping you get your gardening in during shelter-in-place!attack vectors and general confusion as to who is doing what now.

More info on Tolstoy:

Employee Monitoring Trends

Author: Mikey Innovich
Date: Oct 16, 2019
Read time: 1 min

Employers are increasingly using non traditional employee monitoring tools and employees are growing more comfortable with it - if you tell them what you are doing and why.

New tools are being provided to listen to employees in non traditional ways. 
Last year of 239 large corporations surveyed by Gartner, nearly 50% are using some kind of non traditional monitoring software.  That is up from only 30% in 2015, and we expect by 2020, this will be at 80% will montior regularly.

When it comes to non-traditional monitoring, we are referring to analyzing the text of emails, and social media messages, scrutinizing who’s meeting with whom and gathering biometric data while understanding how employees are using their workspace.

Employee surveys have their limits.

They key to an effective monitoring solution within an enterprise lays foundationally in how transparent the organization is as to the how and why.  As long as those are communicated in advance and understood that this is to the benefit of the health of the organization then most of the time this rolls out quite successfully.

4D Film coming to you

Author:  SF Editor
Date: Sept 2019
Read time:  .25 min

Due to entertainment competition with with more and more people spending time in home theater (along with subscription entertainment) are banking on 4D Film, with some local theater betting big on 4D to bring back viewers to the big screen.

So what exactly  is 4D Film?

Quick simply, 4D is a completely immersive experience, complete with synchronized motion seats and live sensory effects like wind and rain and even smell.

In San Francisco, the Korean Movie Theatre chain (CJ CJV ) is betting big on it by setting a stake  in the famous 1000 Van Ness building in downtown San Francisco to replace the previous AMC theatre outpost in that building.    The 20 screen theatre is set to open up in mid 2020.

Traditionally some 4D style cinema was common at some theme parks.  However the new 4D style will be a full upgrade and promises to pull the viewer in like never before.

We are waiting and certainly excited to see this transition from traditional box theatres players.

More to come for sure.  Our staff plans to be at the grand reopening of the 1000 Van Ness Theatre with CJV 4D film. and will provide and update report.

Sometimes it is important to cut it up

Author:  Head Editor
Date: Aug 2019
Read time:  .10 min

Our primary editor went out to quickly moonwalk in the Detroit airport, while reading some texts on his phone. Passerbys were not exactly certain what might be going on, since the moonwalking SVELT editor seemed oblivious to the mini audience watching the smooth flowing moonwalking procedure. Damn straight!

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.