|Last Updated: July 28, 2019|
In this post I'll take you from start to finish in integrating SendGrid to your django project via django-anymail, for using outgoing email and inbound parse to send, receive, and/or forward your custom domain email messages. For free.
The first time I did this is took me about two days because I didn't know when to walk away (more on that in Step 4), and today I did it for the third time and it took under an hour. This post got quite long with a lot more words than code, but I do explain several ways I screwed up the first time, so perhaps you find find it useful if you're stuck trying to use django-anymail with SendGrid to receive email with your django project, and don't know why it's not working.
This post makes the following assumptions:
I don't know who this is for. I don't know if this is a good idea. Let me tell you why I did this:
If this sounds like you, let's start.
After you know how to do this it can be accomplished in less than one hour of work, but it might take 2 days to hit. The first time I did this I drove myself (and the SendGrid support staff) crazy because I didn't walk away and give the MX records time to propagate, so I thought I was doing something wrong. I wasn't. I just wasn't giving it time to take effect. Don't be like me. Follow the instructions to set this up, then chill out and do something else, and come back in 2 days.
Remember how I said I don't send that many emails? SendGrid isn't technically free. SendGrid's pricing is free for the first 30 days (up to 40,000 emails) and then you can then send 100/day, forever. As I said, that's fine with me because these are domains I don't use that often, maybe 100 emails a month, but I just wanted to make sure I'm not being misleading with the word "free".
Make sure you sign up for SendGrid with an email you have access to, so you can verify your account. You don't need it to be the domain you want to send/receive messages from. In fact, if you don't yet have access to mails sent to that account, definately do not use it!
They're gonna email you a link to the email address you gave them when you signed up for an account. Click it.
This post assumes you already know how to add CNAME records to whoever you get your domains from. How to do this will vary depending on who you use, so if you don't know, just google it. I use Namecheap because they're cheap, easy, and their support is decent.
To get the records you'll need to make CNAME records for, go to Sender Authentication from the Settings menu on the left of your SendGrid dashboard. The instructions from Sendgrid are pretty easy to follow, except there is one potential gotcha:
You may need to remove your domain from the record they give you.
For example, If SendGrid gave me a CNAME with the Host of "em9136.teamkaffeeklatsch.com", I'd need to remove the domain (teamkaffeeklatsch.com) and just enter "em9136" for the Host at Namecheap.
In my case this took effect after a minute or two. Just give it a minute, then click "Verify" from the bottom right of the SendGrid screen.
This is also done with your domain host, like Namecheap. And here's where you're going to have to make a decision:
Do you want all of your domain messages sent through SendGrid? Or just a subdomain?
Lets use my domain as an example: teamkaffeeklatsch.com.
If I wanted to continue to get email at another email host, like Zoho or G Suite, and just use SendGrid to receive message from a specific subdomain, I could keep the "@" MX Record for my old host there, and just add the subdomain that I want to use for SendGrid, and give it a different priority from the others. Example:
This way my email@example.com emails are still going to Zoho/G Suite, but firstname.lastname@example.org (or email@example.com) will get picked up by SendGrid's inbound parse. I would do this if I like using the Zoho/G Suite interface (and don't mind paying for it) but also want to automate something with that domain, and want to make it easy to tell my django project which inbound emails it needs to concern itself with (in this case my django app would only receive messages send to the rss.teamkaffeeklatsch.com domain).
But if I want to use django-anymail to forward all messages received by that domain, I'd do this:
This is for the domain I don't use that often, and just want to use my django project at that domain to forward any incoming messages to my gmail inbox.
You could also set up your MX records exclusively with SendGrid, and have SendGrid and your django project receive all messages to your domain, forward you the regular ones (the firstname.lastname@example.org ones) and then do something else with the others
This is for my domain that I get emails from a few times a week and want to be able to respond as a human from, but also want to automate part of the project.
What you need to do after setting your MX records is actually the hardest part of the whole process: go the fuck away and do something else. I'm serious. This is where I wasted 2 days of my life thinking I had done something wrong, when I should have just walked the fuck away and come back to it in 2 days. It'll take 2 days to work. Don't get obsessed with it. Do something else and come back.
You have been warned.
I didn't have any problem installing django-anymail. Their instructions are pretty clear. Make sure that you include the anymail urls into your main projects urls, or else the inbound parse will not work.
This is also easy. Get your API Key from SendGrid.
The only thing to note here is that SendGrid will only ever show you your API Key once, so make sure you're ready to copy it into your settings, when you request it.
Put your Sendgrid API Key in your django project settings, according to django-anymail's instructions. Also easy.
This is strongly recommended, maybe even required, for inbound parse. I already had free HTTPS through LetsEncrypt, so I don't remember how long this takes. I don't know where you have your django project hosted, but PythonAnywhere makes it very easy to use LetsEncrypt with your django app, like they do most things.
Wander back over to your SendGrid dashboard and get to Inbound Parse, via the Settings from the menu on the left. Add the url you want SendGrid to send the messages to.
This part is pretty well explained by the django-anymail installation instructions. They tell you to add
So for example if the random key I had generated were "FGvjzQreVecqsgNi:hgJbGNpPyaRKiq3o" (it's not) my url would be:
When you are adding a new URL with SendGrid, you will only need to add a subdomain if you are using one. In the preceding examples for Step 4, I showed you possible MX configurations that may include the subdomains of "rss" or "parse". If you decided to do that, add that subdomain when prompted by Sendgrid; else, just leave it blank.
There's a couple different ways for this to go wrong. Most of the reasons my code went wrong at first are related to:
If these sounds like stupid mistakes to you (they are) skip ahead to code snippets. If you think you're making them, read on and I'll explain
handle_inbound similar to as indicated in the anymail docs, but it did not work immediately. The first problem I had was that made a new .py file for it, and that code never ran. So I was sending myself emails, and SendGrid was showing the activity, but I was seeing nothing in my inbox (to where I had written code to forward the emails). I had a feeling my new file was the problem, and I didn't feel like re-reading django's signal docs to figure out why, so I just dumped
handle_inbound somewhere hideous where I knew it didn't belong but would nonetheless run, like in a views.py file.
After I moved the the
handle_inbound code to a views file it ran (Hooray!)
After I knew the
handle_inbound code more or less worked I decided to make a new app to handle receiving, forwarding, and parsing emails, so I moved the
handle_inbound code to its own app, in models.py
My subsequent were related to trying to identify the email event, so I wouldn't send myself the same message multiple times. I had high hopes for being able to identify by
event_event_id, but that failed miserably because SendGrid never supplied one, so it was always None (That was a hilarious foible—trying to get object by
event_event_id, which meant that I created a new MessageEvent object one time and one time only).
Then I tried by
event_esp_event but that always looks the same:
WSGIRequest: POST '/anymail/sendgrid/inbound/'. I hadn't read the docs carefully enough and I thought that was a large JSON object, but it was just a tiny string. Once again, I ran into the problem of it only working one time, and then subsequently thinking different emails were already accounted for.
I ended up just using
get_or_create, including several of the fields, and it seems to be working out so far
Another of the problems I had was that I hadn't converted the
from_email to string, which is what is expected by
In models.py of that new app, I made something like this:
And then lower in the models I wrote the following
(NOTE all of the imports are actually at the top of the file, I just split them up to keep near relevant code snippets):
These snippets are a little oversimplified, because I"m not including the code I wrote for the Blacklist or Attachment models, nor for the get_forwarding_email method, but you get the general idea. If you don't get the idea, go ahead and look at the the repo.
This is a super cool piece of advice I got from Nathan H. Leung in his post on Medium. The reason his post is a bonus tip and not the whole thing is because he shows you how to send domain email from through your gmail, but not receive it. I just showed you how to forward your domain mail to an inbox, say your gmail, to receive it, but not send mail. Tada! Put these together and you can have django-anymail forward your domain mail to your gmail, and then you can write back from your gmail UI but have the receiver see only your domain email.
Again, this is why I mentioned it's for domains I don't use too often. It would be quite cumbersome to do all the time (I know it's only a matter of time before I forget to change the sender and accidentally reply from my gmail account) and would not be worth the $50/year it saves in Zoho/G Suite account fees to do it in this circuitous way.
Anyway, to set up your gmail account so you can send mail from your domain, skip to Step 4 of his post.
Basically, what you need to do is:
SMTP Server: smtp.sendgrid.net
Password: your SendGrid Api Key
That's it. Now, when you're writing emails from your gmail inbox, you can choose between your regular gmail address, or the domain one you just added. Cool, yes?
If you don't want SendGrid to think you're a spammer, you can also modify the django admin to send emails. Basically, all you have to do is:
response_changein the model admin
In my case, I did something like this in my email app's admin.py
and then created a new template (changeform.html) in
switchboard_operator/templates/admin/switchboard_operator/messageevent (my app is called switchboard_operator, and the model is MessageEvent)
And now, it's a little clunky, but I can send emails from my domain address out of the django admin.
After I did all this and wrote this entire post I found Django Mail Admin which also lets you send and receive mail via the django admin. I haven't tried it, so I don't know if their implementation is better than mine. ¯\_(ツ)_/¯ I don't think it auto-forwards, which is all I really wanted to do. It looks nicer, though.
Damn this is a long post. Well, if it helped you avoid the same mistakes I did, or if you thought of a better way to identify received emails, go ahead and shoot me an email. I'll probably get it, unless I screwed up my email forwarding. :-P