None

How to Get Free Domain Email with Django and SendGrid
Use django-anymail and an ESP to get free domain email forwarding

Last Updated: Jan. 28, 2019

In this post I'll take you from start to finish in integrating SendGrid to your django project via django-anymail, for using outgoing email and inbound parse to send, receive, and/or forward your custom domain email messages. For free.

The first time I did this is took me about two days because I didn't know when to walk away (more on that in Step 4), and today I did it for the third time and it took under an hour. This post got quite long with a lot more words than code, but I do explain several ways I screwed up the first time, so perhaps you find find it useful if you're stuck trying to use django-anymail with SendGrid to receive email with your django project, and don't know why it's not working.

This post makes the following assumptions:

  • You already know how to use django and have a running project (You don't need to be that good at it, don't worry)
  • You already have a domain and have access to the Control Panel for adding CNAME and MX records.

Who is this for?

I don't know who this is for. I don't know if this is a good idea. Let me tell you why I did this:

  • I have several domains that I use for email for this blog or that website. I want them to look more professional than from a gmail account, but I don't use them often enough to justify paying for G Suite or a Zoho business account.
  • You can set up a free domain with Zoho, but mail forwarding is no longer offered with their free accounts. I don't use these domains often enough to justify checking each inbox every day, but I do want to know when I get messages to them.
  • I had a django project I wanted to integrate daily automated emails into, so it was worth my time to look into django-anymail.

If this sounds like you, let's start.

How to use django-anymail and SendGrid to get free domain email forwarding

After you know how to do this it can be accomplished in less than one hour of work, but it might take 2 days to hit. The first time I did this I drove myself (and the SendGrid support staff) crazy because I didn't walk away and give the MX records time to propagate, so I thought I was doing something wrong. I wasn't. I just wasn't giving it time to take effect. Don't be like me. Follow the instructions to set this up, then chill out and do something else, and come back in 2 days.

General Steps:

  1. Get a free SendGrid account
  2. Verify email address you use with account
  3. Authenticate your domain
  4. Add MX records
  5. Install django-anymail into your django project
  6. Get your SendGrid API Key
  7. Add API Key to django project settings
  8. Get https if you don’t already have it
  9. Add inbound parse url to SendGrid
  10. Add receiver code to your project
  11. BONUS 1: Add SendGrid API Key to your gmail account so you can send domain mails from your inbox as well
  12. BONUS 2: Use the Django admin to send emails

Step by Step

1.Get a free Sendgrid account

Remember how I said I don't send that many emails? SendGrid isn't technically free. SendGrid's pricing is free for the first 30 days (up to 40,000 emails) and then you can then send 100/day, forever. As I said, that's fine with me because these are domains I don't use that often, maybe 100 emails a month, but I just wanted to make sure I'm not being misleading with the word "free".

Make sure you sign up for SendGrid with an email you have access to, so you can verify your account. You don't need it to be the domain you want to send/receive messages from. In fact, if you don't yet have access to mails sent to that account, definately do not use it!

2. Verify email address you use with account.

They're gonna email you a link to the email address you gave them when you signed up for an account. Click it.

3. Authenticate your domain with Sendgrid

This post assumes you already know how to add CNAME records to whoever you get your domains from. How to do this will vary depending on who you use, so if you don't know, just google it. I use Namecheap because they're cheap, easy, and their support is decent.

To get the records you'll need to make CNAME records for, go to Sender Authentication from the Settings menu on the left of your SendGrid dashboard. The instructions from Sendgrid are pretty easy to follow, except there is one potential gotcha:

You may need to remove your domain from the record they give you.

For example, If SendGrid gave me a CNAME with the Host of "em9136.teamkaffeeklatsch.com", I'd need to remove the domain (teamkaffeeklatsch.com) and just enter "em9136" for the Host at Namecheap.

In my case this took effect after a minute or two. Just give it a minute, then click "Verify" from the bottom right of the SendGrid screen.

4. Add MX records

This is also done with your domain host, like Namecheap. And here's where you're going to have to make a decision:

Do you want all of your domain messages sent through SendGrid? Or just a subdomain?

Lets use my domain as an example: teamkaffeeklatsch.com.

If I wanted to continue to get email at another email host, like Zoho or G Suite, and just use SendGrid to receive message from a specific subdomain, I could keep the "@" MX Record for my old host there, and just add the subdomain that I want to use for SendGrid, and give it a different priority from the others. Example:

This way my info@teamkaffeeklatsch.com emails are still going to Zoho/G Suite, but post@rss.teamkaffeeklatsch.com (or whatever@rss.teamkaffeeklatsch.com) will get picked up by SendGrid's inbound parse. I would do this if I like using the Zoho/G Suite interface (and don't mind paying for it) but also want to automate something with that domain, and want to make it easy to tell my django project which inbound emails it needs to concern itself with (in this case my django app would only receive messages send to the rss.teamkaffeeklatsch.com domain).

But if I want to use django-anymail to forward all messages received by that domain, I'd do this:

This is for the domain I don't use that often, and just want to use my django project at that domain to forward any incoming messages to my gmail inbox.

You could also set up your MX records exclusively with SendGrid, and have SendGrid and your django project receive all messages to your domain, forward you the regular ones (the info@teamkaffeeklatsch.com ones) and then do something else with the others

This is for my domain that I get emails from a few times a week and want to be able to respond as a human from, but also want to automate part of the project.

...after you set your MX records...

What you need to do after setting your MX records is actually the hardest part of the whole process: go the fuck away and do something else. I'm serious. This is where I wasted 2 days of my life thinking I had done something wrong, when I should have just walked the fuck away and come back to it in 2 days. It'll take 2 days to work. Don't get obsessed with it. Do something else and come back.

You have been warned.

5. Install django-anymail into your django project

I didn't have any problem installing django-anymail. Their instructions are pretty clear. Make sure that you include the anymail urls into your main projects urls, or else the inbound parse will not work.

6. Get your sendgrid API Key.

This is also easy. Get your API Key from SendGrid.

The only thing to note here is that SendGrid will only ever show you your API Key once, so make sure you're ready to copy it into your settings, when you request it.

7. Add API Key to django project settings.

Put your Sendgrid API Key in your django project settings, according to django-anymail's instructions. Also easy.

8. Get https if you don’t already have it.

This is strongly recommended, maybe even required, for inbound parse. I already had free HTTPS through LetsEncrypt, so I don't remember how long this takes. I don't know where you have your django project hosted, but PythonAnywhere makes it very easy to use LetsEncrypt with your django app, like they do most things.

9. Add inbound parse url to SendGrid

Wander back over to your SendGrid dashboard and get to Inbound Parse, via the Settings from the menu on the left. Add the url you want SendGrid to send the messages to.

This part is pretty well explained by the django-anymail installation instructions. They tell you to add
https://random:random@yoursite.example.com/anymail/esp/type/
So for example if the random key I had generated were "FGvjzQreVecqsgNi:hgJbGNpPyaRKiq3o" (it's not) my url would be:
https://FGvjzQreVecqsgNi:hgJbGNpPyaRKiq3o@www.teamkaffeeklatsch.com/anymail/sendgrid/inbound/

When you are adding a new URL with SendGrid, you will only need to add a subdomain if you are using one. In the preceding examples for Step 4, I showed you possible MX configurations that may include the subdomains of "rss" or "parse". If you decided to do that, add that subdomain when prompted by Sendgrid; else, just leave it blank.

10. Add receiver code to your project

UPDATE: If you don't want to write the code it yourself, I made this into an app

There's a couple different ways for this to go wrong. Most of the reasons my code went wrong at first are related to:

  • I put the receiver code somewhere it never ran
  • while trying to find a way to identify emails, I kept trying to distinguish them by fields that I didn't realize were None
  • I was giving django's EmailMessage class an EmailAddress object, when it should have been given a string, in the to and reply_to arguments.

If these sounds like stupid mistakes to you (they are) skip ahead to code snippets. If you think you're making them, read on and I'll explain

I have not failed. I've just found 10,000 ways that won't work...

I added handle_inbound similar to as indicated in the anymail docs, but it did not work immediately. The first problem I had was that made a new .py file for it, and that code never ran. So I was sending myself emails, and SendGrid was showing the activity, but I was seeing nothing in my inbox (to where I had written code to forward the emails). I had a feeling my new file was the problem, and I didn't feel like re-reading django's signal docs to figure out why, so I just dumped handle_inbound somewhere hideous where I knew it didn't belong but would nonetheless run, like in a views.py file.

After I moved the the handle_inbound code to a views file it ran (Hooray!)

After I knew the handle_inbound code more or less worked I decided to make a new app to handle receiving, forwarding, and parsing emails, so I moved the handle_inbound code to its own app, in models.py

My subsequent were related to trying to identify the email event, so I wouldn't send myself the same message multiple times. I had high hopes for being able to identify by event_event_id, but that failed miserably because SendGrid never supplied one, so it was always None (That was a hilarious foible—trying to get object by event_event_id, which meant that I created a new MessageEvent object one time and one time only).

Then I tried by event_esp_event but that always looks the same: WSGIRequest: POST '/anymail/sendgrid/inbound/'. I hadn't read the docs carefully enough and I thought that was a large JSON object, but it was just a tiny string. Once again, I ran into the problem of it only working one time, and then subsequently thinking different emails were already accounted for.

I ended up just using get_or_create, including several of the fields, and it seems to be working out so far

Another of the problems I had was that I hadn't converted the from_email to string, which is what is expected by EmailMessage. Duh.

Code Snippets

In models.py of that new app, I made something like this:

And then lower in the models I wrote the following
(NOTE all of the imports are actually at the top of the file, I just split them up to keep near relevant code snippets):

These snippets are a little oversimplified, because I"m not including the code I wrote for the Blacklist or Attachment models, nor for the get_forwarding_email method, but you get the general idea. If you don't get the idea, go ahead and look at the the repo.

BONUS 1: Add to gmail so you can send domain mails from your inbox as well

Sending and receiving mail from different places (django app and gmail) raised a flag with SendGrid and got my account suspended for a day, but only 1 day, until we clarified that I'm not a spammer. In other words, this is a great trick but using all of this together can be a little problematic.

This is a super cool piece of advice I got from Nathan H. Leung in his post on Medium. The reason his post is a bonus tip and not the whole thing is because he shows you how to send domain email from through your gmail, but not receive it. I just showed you how to forward your domain mail to an inbox, say your gmail, to receive it, but not send mail. Tada! Put these together and you can have django-anymail forward your domain mail to your gmail, and then you can write back from your gmail UI but have the receiver see only your domain email.

Again, this is why I mentioned it's for domains I don't use too often. It would be quite cumbersome to do all the time (I know it's only a matter of time before I forget to change the sender and accidentally reply from my gmail account) and would not be worth the $50/year it saves in Zoho/G Suite account fees to do it in this circuitous way.

Anyway, to set up your gmail account so you can send mail from your domain, skip to Step 4 of his post.

Basically, what you need to do is:

  1. Get to “Settings” of your Gmail inbox.
  2. On the Settings screen, click the “Accounts and Import”
  3. In the “Send mail as:” section, click the “Add another email address” link. Add your email address, one of the ones you set up forwarding from.
  4. Uncheck “Treat as an alias.”
  5. Type SendGrid data into their corresponding fields.:
    SMTP Server: smtp.sendgrid.net
    Username: apikey
    Password: your SendGrid Api Key

That's it. Now, when you're writing emails from your gmail inbox, you can choose between your regular gmail address, or the domain one you just added. Cool, yes?

BONUS 2: Use the Django admin to send emails

If you don't want SendGrid to think you're a spammer, you can also modify the django admin to send emails. Basically, all you have to do is:

  • override response_change in the model admin
  • override the change_form.html template

In my case, I did something like this in my email app's admin.py

and then created a new template (changeform.html) in switchboard_operator/templates/admin/switchboard_operator/messageevent (my app is called switchboard_operator, and the model is MessageEvent)

And now, it's a little clunky, but I can send emails from my domain address out of the django admin.

After I did all this and wrote this entire post I found Django Mail Admin which also lets you send and receive mail via the django admin. I haven't tried it, so I don't know if their implementation is better than mine. ¯\_(ツ)_/¯ I don't think it auto-forwards, which is all I really wanted to do. It looks nicer, though.

End

Damn this is a long post. Well, if it helped you avoid the same mistakes I did, or if you thought of a better way to identify received emails, go ahead and shoot me an email. I'll probably get it, unless I screwed up my email forwarding. :-P

Connect With Us


info@teamkaffeeklatsch.com