An email odyssey: how we tamed the bit of the app no one wants to work on

This post was written by Jack Weeden. Jack's an engineer at Lost My Name and was the Tech Lead on this project.

We send a lot of email. Between transactional and behavioural emails we send around 60,000 emails per month (plus our regular newsletter). These emails tell customers vital information about their orders, and are an important part of our sales and marketing operation.

Email is clearly important. Unfortunately, email is also hard. Hard to work with as a developer and hard to understand and manage as a non-technical person, especially for an internationalised business like ours that operates in several languages.

Anyone working in a SaaS or Ecommerce company will probably be able to relate the challenge of email. Because its hard, it rarely gets solved well, and all round the world people are struggling with interlinked, gerry rigged, hard to visualise, hard to understand email templating / sending / reporting systems.

We had one of these at Lost My Name until recently. In this post I'm going to explain our journey from there to where we are today - operating a simple, easy to manage, easy to test custom email system that handles internationalised content easily.

In this post I'll outline:

  • The key types of email, transactional, behavioural, newsletters and internal

  • Our initial setup that came with our Ruby on Rails Ecommerce platform Spree and what was broken about it

  • Our experiment with 3rd party providers especially customer.io and what was broken about them

  • How we came to design our own system, Koala, how it works and why our email system is no longer broken

Hopefully this post will be useful to you in your work. We'd love to get your feedback via twitter @lostmynameHQ

The email landscape

There are many SaaS providers offering email delivery solutions, from direct SMTP delivery like Mandrill to more developer-focused API approaches like Mailgun, leaving you as a developer to concentrate on more domain-specific - and hopefully more interesting - problems.We group our emails into four categories: transactional, behavioural, newsletter and internal.

Emails are broadly defined by their delivery triggers and intent.

1 Transactional

Transactional emails are event-triggered, consequences of a change in status of an order or a customer performing a specific action. Currently we send the following:

  • Order success: containing a summary of a specific order along with estimated delivery time
  • Book printing: we have a short grace period between order completion and the book being printed. This allows any mistakes to be rectified before committing the book to print
  • Order dispatched: letting the customer know their book has been printed and is on its way. This contains a link to track their order with an external delivery service
  • Password reset: a standard password recovery email for when a user forgets their password

Transactional emails should typically have a very high open rate because they contain information directly relating to an order a customer has placed or they are delivered in response to an action the customer has performed. Furthermore, a customer shouldn't be able to unsubscribe from these emails and they should be delivered immediately following the trigger event.

2 Behavioural

Behavioural emails are a mixture of event-triggered and time-based emails. They are primarily related to our referral program and in general are typically more marketing-driven deliveries aimed at conversion and customer retention.

  • Referral introduction: information about our referral program. We aim to deliver this just after the customer has received their order. Depending on their country of origin this differs from customer to customer
  • Referral updates: when a customer has referred a friend and that friend has bought a book, we let the original customer with an email
  • Referral success: when a customer has referred three friends, they are eligible a free book not available to purchase through the site. This email allows them to claim the book
  • Abandoned basket: if a new customer abandons their order part way through, we send a prompt containing the order information and a direct link to pick up from where they left off

3 Newsletters

We send out a montly newsletter to our customers with news, events, competitions and so on. These are authored by our marketing team and can be sent to a certain segment of customers depending on the content.###

4 Internal

Like many tech companies, we have background tasks which are run periodically. Certain people in the company need to be notified when a task has run successfully or has failed for some reason, so we have a number of jobs which trigger internal emails. These are typically plain text or sometimes are totally blank, only containing an attachment.

Our email journey begins...

We rebuilt our main app about a year ago around the open-source e-commerce platform Spree. We weighed up a few different options and settled on Spree as it's a fairly mature open source platform. Although we don't conform to your typical e-commerce company (our products are personalised so we don't stock any inventory), Spree takes care of a lot of things we would have had to build ourselves, allowing us to focus on making our product and user experience as great as possible.

Our initial transactional setup

While Spree is an open source project, it does have commercial backing, funded primarily by the sale of Wombat, its "operating system for commerce". This is essentially a queueing system connecting your Spree store front to any number of third-party integrations. The idea is you push your store data (orders, customers, shipments) to Wombat and it'll pipe that data to any integrations you have set up. In order for the data to be in the correct format for the integration, you define a transform in which you manipulate the data before you pass it off. One of Wombat's primary selling points is that its point-and-click interface allows non-developers to make changes, and changes to be made without app deployments. Need the subject line of an email changing? Just log in and edit the transform.

This was an attractive feature for us so we began sending our transactional emails using Wombat as it was quick to set up and relatively easy to maintain. The fact it's designed primarily to be used with Spree was also a contributing factor. Using Wombat, the flow for sending transactional emails was as follows:

  • 1. Serlialise the order, shipment or user object and send it to Wombat
  • 2. Transform the order data into a format Mandrill understands
  • 3. Deliver the email using the appropriate email template in Mandrill, injecting in data from step 2

This was quite a simple approach and worked well to begin with, however problems arose as our product, and therefore main app, got more complex. The first issue was the transform, which quickly spiralled out of control. Changing it evolved from a job a non-developer could do to a mess of code incomprehensible to most of our developers.

The second issue was internationalisation. We had a nice HTML template set up in Mandrill for sending these emails, but when we launched the book in our first non-English language, we had to create another template. This in turn meant adding conditional logic in the transform in order to select the correct template based on the customer's locale.

Creating a new order confirmation email for a new locale therefore involved copying the entire HTML content from an existing email, gathering the new translations, overwriting the existing copy, and pasting it into a new template. Anyone who has dealt with email template HTML before knows what a mess the markup looks like:

Furthermore, a small change in the template meant changing it in every other template. This had the effect of developers being reluctant to make small copy changes. When we came to launch our fourth language, we'd had enough of this approach so we started to think about how we could change it.

Enter Rails...

In the end we settled on what would seem to be an obvious solution: send emails straight from our main Rails app. This comes with many advantages: * Emails templates are just normal ERB templates, so common parts can be refactored into partials and used across multiple templates. Additionally it means we can abstract away all of the verbose HTML markup into nice Rails helpers. * Translating the copy can be done in the same place as the copy on the rest of the site. Our team of translators use an external service. They're familiar with it, and it makes sense to keep all translations in the same place. * Rails is good at email. We chose to use ActionMailer but similarly we could have farmed off the actual sending mechanism to any third party service. * Emails are sent asychronously by pushing jobs onto our Sidekiq queue. This has two great upsides: * We already have mechanisms set up to detect when Sidekiq jobs aren't being processed, or they raise exceptions, so we get all of this for free by using Sidekiq for our email delivery. By using a third party service, we don't get visibility of this kind of stuff. * We can assign priorities to our queues. The reset password email should be delivered to a user as soon as possible after they request it, so this gets a higher priority over our referral program emails.

Using our main app does have its downsides though, too.

  • We're adding complexity to an already quite large monolithic system. Using Wombat, we could just ship off the serialized object and forget about it. Now we've got this logic embedded in our e-commerce app.
  • Third party services built around sending email typically have guards against multiple redelivery, ensuring people don't get sent the same email over and over. We had to build this functionality ourselves.
  • Editing templates now requires the work of a developer and an app deploy to take effect. This is quite a big disadvantage, but we tried to minimise its imact as much as possible. Our international translators are able to change the wording of translations and deploy them using Slack. This means copy changes are quick and easy to implement. Changing the actual styling or structure of a template still requires developer resource though.
  • Images are a bit of a pain to work with. By default, the Rails asset pipeline appends a unique hash to an image filename based on its contents. So if an image changes, its filename changes as well, meaning that older emails sent out before the file changed would no longer be able to render any of the images. We have plans to get around this in the future but for now email images are uploaded manually to S3 and referenced directly in the templates.

Overall we decided the pros far outweighed the cons and all of our transactional emails are now sent from our main app. ActionMailer is set up to perform the actual delivery using the Mandrill SMTP server.

Our first attempt at behavioural email

Customer IO

Just like with transactional emails, we wanted a system that could be used by non-developers. This was especially important with behavioural emails where we are trying to work towards some sort of conversion goal, not just presenting information relating to an order or shipment. This meant we wanted a solution where A/B tests could be implemented and measured, conversion goals tracked and customers could be segmented to target customers on a granular level.We shopped around a bit and finally settled on using Customer IO, which seemed to meet the majority of our needs.

The premise was similar to using Wombat for our transactional email: we would send relevant data to Customer IO throughout a customer's purchase journey and beyond, then let it take care of the rest. We could go into Customer IO, set up triggered campaigns based on specific customer attributes and we could set up our email campaigns that way.

We trialled Customer IO for our referral program and it worked well. We knew what data we needed to send in order to segment users the way we wanted, so we quickly implemented that in our main app.

Soon enough we had populated Customer IO with enough data about new customers to start sending our campaign emails. Members of our marketing team were able to set up campaigns, target them at specific users - e.g. a certain locale, number of previous orders, etc. - and could get a nice overview of the send, open and click rates as well as conversions.

This was very successful to start with and ran smoothly for several weeks; we trialled A/B testing email subject lines and set conversion goals. Everything was working nicely so we decided to set up another campaign directed at a subset of users we hadn't originally anticipated. This raised the issue that we didn't have the required data stored in Customer IO, so we had to write a small script to pick out data from our main app's database and update all users in Customer IO using their API. The API was rate limited so updating thousands of customer records took a significant amount of time.

We realised that unless we essentially mirrored our customer table in Customer IO, we'd continually be having to push up new attributes using the API. Even then, we might want to segment customers based on a combination of data points, e.g. customers who bought one book for a boy and one for a girl, or customers who bought 3 books within a certain date range. We'd have to push new data up for every customer for each new campaign. Unfortunately it became obvious that this wouldn't be a viable options either.

So, just like with our transactional emails, we opted to roll these emails into our main app. This gave us most of the advantages as before: we could use the same templates, internationalisation came for free and more visibility around errors and failed sends.

What about Newsletters?!

Mailchimp is a very popular email marketing service and we've been using it to send our monthly newsletters for some time now. It provides a nice WYSIWYG interface for composing a newsletter and delivering it to a customer base and, after initial template creation, it's very easy for anybody to use. Because of the complex authoring tools you get with Mailchimp, we decided not to roll our own version just yet.

Koala

The major downside of switching to our own system for email delivery is that we lost out on the nice analytics that third party systems typically provide. Analytics, and especially conversion rates, are very important to Lost My Name; we like to know what's going on and, even moreso, being able to dig around in our datasets. We were still using Mandrill under the hood to actually deliver the emails and Mandrill does a nice job of showing analytics for a short period of time, but we decided we could do a bit more, especially around conversion goals. So we made a small Rails service, Koala, to collect and analyse our email data.

Mandrill provides a nice webhook which it'll ping each time an even occurs - sends, opens, clicks, bounces, etc. We built a very simple app which listens to these webhook events and populates a database. While this part mimicks Mandrill's own analytics, we can store data indefinitely and mine it however we want.

Because Koala is being Slack authentication, anyone in the company is able to go in and get an overview of how our emails are performing. Additionally, because the data is persisted to a PostgreSQL database, we can hook it up Periscope, which means anyone can run queries against the data. If we want know how many users have unsubscribed in the past month, we can simply write an SQL query and Periscope will chart it.

Being fully in control of the data and how we store it also means we can augment it with data from other sources. In our main app we can make an API call to Koala, for example, to add converstion data to an email. We then get more visibility of the effectiveness of a certain campaign.

A bonus feature is that because Mailchimp sends its emails using Mandrill under the hood, we get all the data around our newsletter emails pumped straight into Koala for free.

An email history

We often get requests from our customer support team to regenerate an email for a customer if they've lost or deleted it. This used to be a bit of an involved process - we'd have to open up a console on Heroku, paste in a few commands, copy the HTML generated, paste it into a new document, take a screenshot and send it back to the customer. With Koala, we decided fairly early on that we'd like this process to either be automated or, even better, the customer support team would just be able to view all sent emails. Now every time an email is delivered, we store the HTML content in Amazon S3 and make this available through Koala, making the process of troubleshooting any past email deliveries a breeze. As a bonus, we made a Zendesk integration allowing our customer service team to see which emails have been delivered to a particular customer from within Zendesk.

A simplified architecture

Overall, we've greatly simplified our email architecture and in the process of doing so created a simple but powerful analytics platform. It was important for us to shop around and try out various third party services before jumping straight in an rolling our own solutions - we're big proponents of using tried and tested alternatives before reinventing the wheel. Having so many moving parts and our data duplicated in many different places meant we had to stop and evaluate our approach. In doing so we were able to simplify our existing system, which looked like this:

We ended up with a system which looks like the following:

It is now easier to understand, easier to change and easier to test email than ever before and generally it makes everyone in the business much more confident about the performance of our email systems.

It was a long journey to get here, but we're confident we've now built a foundation that will enable our email ambitions to grow with our company over the next 12 months.

As I mentioned before, I'd love to get your feedback and hear about how you tame the email beast at your startup on Twitter - ping us @lostmynameHQ

Join us!

If you’d like to come and work, play and hang out with us, here’s what we’re looking for. Though you could just send us something phenomenal, leaving us with no choice but to follow up, to:

careers@lostmy.name

Financial Controller

We’re looking for an experienced Financial Controller to oversee the global reporting and compliance functions, leading a team of two finance analysts to deliver continuous improvement in the financial operating model.

Reporting into the CFO, the successful candidate will demonstrate a high degree of commercial acumen and be comfortable working in a fast-paced and dynamic start-up setting where every week will present new opportunities and challenges to navigate.

With ample room to grow, this role is a great chance for the right candidate to work alongside the leadership team and deliver tangible improvements for the business allowing us to deliver on our 3 year strategic plan.

The key areas of focus are:

Reporting

  • Ownership of the financial month end close process for Lostmy.name Ltd and Lostmy.name Inc
  • Consolidation accounting for Lostmy.name Group (UK and US Entity)
  • Preparation of month end commentary packs and presentation of results to the senior leadership team together with key action points
  • Cash flow and working capital management

Compliance & Control

  • Overseeing global tax, regulatory and compliance agenda including Sales, Income and Payroll Taxes in the UK and US
  • Owning the year end external financial statement audit process
  • Laying down a robust control environment in order to deliver accurate financials with a high level of data integrity

Business Partnering

  • Leading cost saving initiatives to help the business to improve operating efficiency
  • Working closely with the BI team in order to streamline and automate the reporting process
  • Working closely with the engineering team on the R&D tax credit workflow

Requirements

  • Qualified accountant (minimum 2 years PQE in an industry position) - ACA, ACCA, CIMA
  • Experience managing a team and overseeing a financial month end close process for a group
  • Experience with US accounting/tax is a plus
  • Willingness to roll up sleeves and get stuck into the detail while simultaneously keeping an eye on the big picture
  • Self- starter, high degree of emotional intelligence and comfortable making decisions where there may be a high degree of uncertainty
  • Experience using cloud accounting software; we use Xero in combination with receipt bank. Experience of these packages is a plus but not a requirement
  • Experience in start-ups/tech/publishing/retail/consumer products is a plus but again not a hard requirement

Benefits

  • Competitive salary and a real stake in the business - you'll share our success
  • Be part of a fast growing company creating products that are really loved by customers all round the world
  • Manage your own work as part of a small, tightly knit team
  • Flexible office hours
  • Awesome BUPA healthcare for you and your whole family
  • Fantastic mobile phone scheme & childcare vouchers benefits
  • Training budget of £1000: we'll sponsor any conference you wish to attend or speak at
  • Delicious team lunches

Senior Copywriter

Words are brilliant. But there are millions of them, and without good writers, words just sit there in the wrong order saying the wrong things. Rotters.

We’re looking for a remarkably creative comms writer who can make words do exactly (exactly!) what they need them to: entertain, surprise, jump through hoops, play the cello — anything they command. Someone with the ability to be playful and hilarious, who isn’t afraid of a good pun, but who is also marvellously disciplined and hard-working. And they should be brilliant at ideas too.

Feeling confident?

You should have a lot of experience writing comms. You have the knack to be funny and clever without being self-indulgent. You have a light touch but can also land a message with real weight. You know how to edit yourself, how to think about messaging hierarchy, how to make people open emails, click buttons and buy products. And you should be able to create funny, sharable content that your mum might see and chuckle at before boasting to friends that you came up with it. You should also be self-motivated, pro-active, collaborative and self-sufficient. You should know how to write for email, social media or on the side of a banana.

Most importantly, you can make people smile every time they read your words or ideas.

Lost My Name is one of Britain’s fastest-growing brands and this is an opportunity to help turn a successful start-up into a global name. We are a playful, kind brand. This opportunity will be ridiculously exciting to a smart, funny, eloquent, engaging, intelligent, autonomous, experienced, creative and all-round superb candidate.

The team you’ll be working in delivers creative output at a higher-frequency than any other team in the business. We literally ship handfuls of content every week and watch and learn from the public reaction. We get to directly shape and influence the brand by continuously exploring and publishing new concepts and ideas.

This role is the sole copywriter in the team, which gives you a central role in expressing the tone of the brand. You will report to a very experienced writer, who can help you sharpen your talent even more.

Key areas of focus

  • Writing emails and blog posts for our weekly Editorial content
  • Writing humourous, sharable content for Twitter & Facebook
  • Collaborating with our designers to devise and develop creative editorial concepts
  • Writing pithy, dad-joke-esque comments on our Instagram content
  • Writing emails for product promotions
  • Helping to firm up our tone of voice for Twitter and Pinterest channels
  • Helping to champion and promote the brand tone of voice across the business

Experience

  • 5+ years’ experience as a copywriter for well known and/or playful/entertaining consumer-facing brands
  • 3+ years of experience with SEO, email, web copy
  • Ability to produce both long-form and short-form writing
  • Extensive experience and understanding of all social channels, including behaviours and algorithms

Benefits

  • Competitive salary and a real stake in the business - you'll share our success
  • Be part of a fast growing company creating products that are really loved by customers all round the world
  • Manage your own work as part of a small, tightly knit team
  • Flexible office hours
  • Awesome BUPA healthcare for you and your whole family
  • Fantastic mobile phone scheme, childcare vouchers benefits
  • Training budget of £1000: we'll sponsor any conference you wish to attend or speak at. Travel the world and share your knowledge on us!
  • Delicious team lunches

Japanese Speaking Customer Service Angel

We're looking for someone to help us share the Lost My Name magic with our friends in Japan! We’re looking for a brilliant Customer Service Angel to help support our increasing presence in Japan. What kinds of things would you get up to in this fun, flexible role? We are so glad that you asked...

Customer Service, mostly...

People approach us with all kinds of queries. Sometimes they just want to know how much shipping is to Tokyo (it's FREE!). Other times, a book has gone on a different route in the mail system and we need to re-create it ASAP in time for a birthday party. Occasionally a customer would like to change a shipping address, or add a personal dedication.

We also get an incredible amount of feedback with ideas for improvement and future products, and just lots and lots of heartfelt thanks for such a wonderful book. These are just a handful of examples of the issues and topics we deal with on a daily basis. And we like to answer and solve ALL of these inquiries in the same way: super responsive, polite, empathic and delightful. We handle our customer support through a collaborative online platform where you can respond, solve, escalate, re-assign, tag and archive all queries in an efficient and transparent way. If you know how to use Facebook or Gmail, you’ll have no problems with our system.

What else should you know?

The role is a remote working opportunity, with some time flexibility to suit you - e.g. mornings, evenings and weekends. There’ll be an induction period with our marvellous training and co-ordinating team, along with a detailed explanation of how it all works.

So do you think that you can help us? If so, then click Apply Now.

  • You got a flair for friendly and positive digital communication - you know how to be nice, and genuinely lovely online
  • Fluency in Japanese and English language (written and spoken) essential
  • You’re looking for a genuinely exciting and meaningful new adventure, and the idea of making children more curious and kind sounds awesome
  • You must be available for at least 20 hours per week (flexible, including weekends)
  • You’re a super motivated, dedicated, smart and an energetic self-starter
  • Ability to work under pressure and adapt quickly to adverse situations

Lead Designer

You’ll have a real passion for design and communication, and a genuine desire to do brilliant work. As a Lead Graphic Designer you will be responsible for taking the lead on the creative development of design projects across multiple platforms. Driving the creative standards of the Design Team and developing junior designers’ creative abilities, making sure the creative brief is fully explored and delivered to the highest standards.

Key Responsibilities

  • Responsible for the design and production of graphic design across multiple platforms.
  • Responsible for working with internal clients to deliver requirements on time and in line with the brand style.
  • Responsible for applying and co-ordinating the design across the brand.
  • Use a good awareness of design processes to question and challenge design briefs and solve design problems.
  • Be aware of current design trends and attitudes to help inspire the team
  • Understanding peripheral processes, i.e. print, interactive and mobile platforms.
  • Manage and develop junior designers.
  • Work closely with the studio manager to manage and prioritise the workload on a project.

About You

  • Ability to deliver against tight and complex timelines.
  • Knowledge of and sensitivity to trends influencing design in general.
  • Ability to manage multiple and competing work priorities, demands.
  • Strong written and oral communication skills, including ability to make persuasive presentations to internal audiences.
  • Driven and passionate.
  • Stress resistant and confident.

Qualifications and Experience:

  • Recognised design degree.
  • Significant industry experience.
  • Versatile and open mined.
  • Meticulous crafting and attention to detail.

Skills:

  • Excellent communication and presentation skills.
  • Dedicated, passionate and challenging.
  • Ability to lead, manage and inspire.
  • Strong ideas with a portfolio to match.
  • Self motivated and a team player.
  • Loves discipline of identity and branding.
  • Excellent working knowledge of Adobe Suite: Photoshop, Illustrator, InDesign, Acrobat

Senior Brand Manager - Books

We’re looking for an exceptional product / brand manager to join our team and look after the ongoing growth of our book portfolio across territories, formats and licensing opportunities. This important role includes taking stewardship of the growth roadmap for our flagship title Lost My Name.

This role is similar to a licensing manager position within a publishing firm but with two vital differences.

The first difference is that you’ll be working within a very agile, vertically integrated business that currently does all it’s own retail, marketing, manufacturing and product design with a sales footprint in every major market in the world.

This means that if you have a plan to achieve growth through change to any of these functions you can make that change happen within days or weeks, not the usual timescales of months or years.

The second difference is that all the brands and products you will manage use personalisation and technology in creative ways to tell their stories. This means that you can have a one to one relationship with customers through the products you market to them - but it also means you face interesting challenges in doing traditional licensing deals.

If these opportunities and challenges excite you, and if you have a track record in building entertainment brands, particularly in the kids and families space, we’d love to hear from you.

Key responsibilities and targets

  • Drive incremental revenue growth in our book range
  • Key stakeholder in setting and driving overall unit targets across territories
  • Own the internationalisation roadmap for all our books
  • Own the format roadmap for all our books
  • Own the individual brand roadmaps for our most successful books, including developing new upsell and retention products via in house design or licensing
  • Line manage several junior delivery colleagues within the product team

Expected experience and skills

  • Business focused and experienced - You have a demonstrable track record growing revenue on entertainment oriented properties. We’d like to meet people who’ve managed and grown products and brands turning over more than £10M a year.
  • Analytical and data driven - you can show us through examples of how you’ve used quantitative and qualitative data to make the right decisions for the business.
  • Creatively sympathetic - you can also show us examples of how you’ve reasoned out plans and projects from a creative position and protected and enhanced creative value in the properties you’ve managed.
  • Strong communicator and leader - you know how to make change happen in a fast moving company and you’ve got experience of directly line managing more junior colleagues.
  • Gets stuff done - above all, we want someone who is going to roll up their sleeves and get stuck into the fantastic challenge of bringing the values of our books to as many people as possible across the world. Lost My Name is a young company that is just getting started on it's journey - You’ll be super excited to join in and start shipping from day one.
See more