Google Apps 'From' Address Spoofing / 'From' Address Override [Update]

Please be sure to read the important update at the end of this article

Alright, so I was just working on a client's project that has a very typical request that always seems to be an issue when using GMail / Google Apps as an integration piece.

And I stumbled upon a solution that although doesn't solve all cases - I'm sure will help a couple of people out.

So here's the dilemma.

Let's say you have a integration solution that includes an e-mail gateway, in this kind of setup.

(Application A) <----> (SMTP/POP/IMAP Server) <---> (E-mail Gateway) <----> (Application B)

So this situation is that you have an application (Application A) which ultimately wants to get a message to another application (Application B). And a lot of the times, the way they do this is through a E-mail Gateway (in other words an application that is able to receive and send e-mail on behalf of Application B). Further to this, perhaps Application B is really a multitude of destinations differentiated by the "To:" e-mailing address in the original message.

Here's an example

(Application A) - is an e-mail generator.
(SMTP/POP/IMAP Server) -
(E-mail Gateway) - is an application that maps the "To:" e-mailing address to a smartphone app user.id
(Application B) - is a smartphone app, where each user has a user.id

So the series of steps would go this way:

- User of (Application A) generates an e-mail to Johnny@hello.com
- (SMTP/POP/IMAP Server) is setup with a catchall@hello.com and will catch all the e-mails that don't have real addresses like Johnny@hello.com. So the original e-mail ends up in the catchall@hello.com inbox.
- (E-mail Gateway) periodically checks the (SMTP/POP/IMAP Server) for e-mails and finds the e-mail meant for Johnny@hello.com. The (E-mail Gateway) maps the "Johnny" part of Johnny@hello.com to a user.id "Johnny" and send a message to (Application B) running on Johnny's smartphone.
- Johnny on his smartphone replies to the message.
- (E-mail Gateway) gets Johnny's reply, and sends the e-mail using the catchall@hello.com account back to the originator (Application A).

Alright so this seems all good except for one piece. No matter who (Application A) sends the e-mail to, even though it will get to the right smartphone, the replies will always come from catchall@hello.com. Well, I shouldn't say that - if you use a GMail or Google Apps for Business GMail account - the from address will always be from catchall@hello.com.

This is because the e-mail gateway can only hook up to the one account and GMail and Google Apps for business (rightly so) prevents overriding/spoofing of the From address.

Now in some instances, this may be fine - the from address won't matter. But in the particular case that I was dealing with, (Application A) would reject the response if it didn't match the original e-mail. So if (Application A) sent out an e-mail to "Johnny@hello.com", it would only accept responses that came back from "Johnny@hello.com".

And the main issue here is that GMail / Google Apps for Business don't allow you to override the From address. So if your E-mail Gateway tries to set From address - GMail will prevent it.

Solution

Well here's the thing - yes, in most cases, if you adopt this kind of solution, GMail and Google Apps for Business will prevent setting the from address.

But there is a way around it if you

1. legitimately own the addresses you are sending to (or the domain of those e-mail addresses) and 
2. are willing to pay for Google Apps (which if it's for a client, hopefully $50/year is not bad)

This situation will actually allow you to use just one e-mail account - but have multiple "From" addresses.

So this situation is:

- (Application A) wants to generate e-mails for Johnny@hello.com, Max@hello.com, Tiffany@hello.com, and Kelly@hello.com.
- And, it will only accept responses from those e-mail addresses.

Here are the 4 key things you need to do:

1. Use Google Apps for Business (yes, yes, it's always annoying when you come across a post that requires you pay money, but again - this may help out some peeps)
2. Create a default account
3. Add nicknames (or aliases)
4. Setup "Send mail as..." for those aliases

So let's say you own hello.com.

1) Setup Google Apps for Business with this domain and

2) create a default account like receiver@hello.com.

Please refer to the update as step 3 is not really valid if you need more than 30 nicknames.
3) Thirdly, from the Admin of your Google Apps (admin.google.com), where you setup your receiver@hello.com account, add your nicknames to the receiver@hello.com

• Johnny@hello.com
• Max@hello.com
• Tiffany@hello.com
• Kelly@hello.com

Alright - so that's the first step. What that step does is allow for incoming mail to all end up in the same box so that your (E-mail Gateway) can poll that single inbox to get all the e-mail.

Alright - so that's not the trick to this solution - because that part usually people have figured out how it works. Do note though - that I'm not really using a receiver@hello.com as a catchall account - catchall accounts usually trap all e-mail that does not have an associated account with it. Rather, I'm using aliases - for an account. This is one of the keys to the success.

4) And here we go - the clincher that makes all this work -

SETUP YOUR "SEND MAIL AS..." e-mail addresses.

So log into the regular Mail with receiver@hello.com and go to your mail account settings (click on the little Gear and go to settings.) Make sure you're in your Mail settings, not your overall google account settings.

Then, hit up the "Accounts" tab and you'll see a section called "Send mail as...". This will be a little bit of a tedious process - but add each of the nicknames to this list:

• Johnny@hello.com
• Max@hello.com
• Tiffany@hello.com
• Kelly@hello.com

You'll have to verify each one with a code. When you add one of those e-mail addresses, Google will send you an e-mail (which will end up in the receiver@hello.com inbox) with a verification code. Just verify that code and leave the rest of the settings as is.

And that's it.

This is the key - you can "spoof" - not really spoof - "override" the "From" e-mail address from a piece of software with addresses that appear in the "Send mail as..." list.

Usually the issue that programmers run into is no matter what they set the "From" address to, if they use a Google Mail account - it will always be from that account.

This way it opens up a bit to a set number of addresses.

With this setup up, from the single receiver@hello.com account, I can send e-mails (programmatically and from Gmail itself) from

• Johnny@teldio.com
• Max@teldio.com
• Tiffany@hello.com
• Kelly@hello.com

So again - I know it's not as open as being able to send from absolutely any address - but it does open up the door for a lot of integrations. I myself have dealt with a lot of integrations that are looking for this exact solution.

Hope this helps out some peeps - if it's confusing at all, feel free to leave a comment and I'll get back to you asap.

[Update]

Alright - so there is an important update to this solution. It turns out that Google Apps / GMail actually puts a cap of 30 nicknames / aliases on each account. And our problem is that we had more than that.

So what's the solution - well it's to half do what I explained above and half not.

1. Setup a single account to be your catch-all account for Incoming E-mail (rather than using nicknames and aliases)
2. Still setup the "Send mail as..." accounts on that catch-all account.

The key here is that there seems to be no limit on the "Send mail as..." - so you can still send on behalf of all those accounts.

And instead of using nicknames / aliases - just use a catch-all account so that all your e-mails will end up in that account.

Ruby on Rails, Recaptcha, AJAX

(excuse the formatting of this post, I can't be bothered right now to deal with the Blogger rich-text editor:P)

So I launched www.capoapp.ca and www.bitesite.ca over a year ago and it was only a matter of time before bot spammers started hitting my website. I quickly started seeing all these nonsense e-mail notifications from people filling out my feedback and contact forms among other public forms.

Without even knowing they were bots, my first attempt to stop these spammers was using a Captcha solution. So I decided to use the www.recaptcha.com and more specifically, the recpatcha gem found here.

I used this on www.capoapp.ca and luckily, within days, the spam stopped. It looks like I had found my solution.

So I implemented recaptcha on www.capoapp.ca's Feedback page and Submit an Open Mic page and all was good.

My next step was to implement it for my company's corporate website, www.bitesite.ca.

I added the recatpcha_tags as instructed, and also added the verify_recaptcha code to the server-side controller (just like I did with www.capoapp.ca), but to no avail.

Capo was working perfectly fine but BiteSite wasn't. What gives?

The problem was actually quite obvious (I must have just not slept well - well actually I was coding BiteSite's solution when I was 12-hours jet-lagged). And while the problem was obvious, the solution wasn't really.

What was the problem? Well it had to do with the submission of the form. In Capo, I was submitting the form by posting the page to the server - and because the recatpcha_tags were part of the form, they automatically went with the form to the controller - and all the necessary components were there for verify_recpatcha to process.

BiteSite on the other hand, I was submitting the form via a JQuery AJAX call. So the fields in my form aren't automatically submitted. In the past, here's what my BiteSite contact form submit AJAX code looked like:

var first_name = $("#first_name").val();
var last_name = $("#last_name").val();
var email_address = $("#email_address").val();
var message = $("#message").val();
$.ajax({
  url:"/contact",
  type: "POST",
  data: { first_name : first_name,
          last_name : last_name,
          email_address : email_address,
          message : message },
  dataType: "json",
  success:function(data, textStatus, jqXHR){
    ...
  },
  error:function(jqXHR, textStatus, errorThrown){
  ...
  }
});

So as you can see, I'm manually passing first_name, last_name etc. to my controller. The issue is that for verify_recaptcha, I need to pass what the user entered for the recaptcha_tags. On top of that, I need to pass any additional information that verify_recaptcha needs to do its work.




So that was the problem?






What's the solution? I need to figure out exactly what to pass to my controller so that verify_recaptcha will work. So luckily, the recatpcha gem is open-source, and I just browsed the code to find that verify_recatpcha relies on 2 main parameters:




params[:recaptcha_challenge_field]

params[:recaptcha_response_field]




So that was it. My Ajax call was missing these two parameters, so all I had to do was change my code to this:



var first_name = $("#first_name").val();

var last_name = $("#last_name").val();

var email_address = $("#email_address").val();

var message = $("#message").val();

var recaptcha_response_field = $("#recaptcha_response_field").val();

var recaptcha_challenge_field = $("#recaptcha_challenge_field").val();



$.ajax({

  url:"/contact",

  type: "POST",

  data: { first_name : first_name, 

          last_name : last_name, 

          email_address : email_address, 

          message : message,

          recaptcha_challenge_field : recaptcha_challenge_field,

          recaptcha_response_field : recaptcha_response_field },

  dataType: "json",

  success:function(data, textStatus, jqXHR){

    ...

  },

  error:function(jqXHR, textStatus, errorThrown){

    ...

  }

});





So that was it. Now it all works fine. Take that bots! (shouldn't egg on the spammers, they're way better programmers than I am and now they're gonna attack me).




Hope that helps out some peeps.

Nikon D7000 Err Message Mirror Stuck

Alright, really quick. I was using my Nikon D7000 for a great shoot and everything was going great until at one point the battery ran out while I was shooting video in Live View and then boom "Err" message.

This came up because my mirror was stuck. Lots of people say you need to send it back to Nikon, but I plugged in a new battery, and shot a picture (to reset the mirror) and all was good.

So if you get this message and your mirror is stuck, just try putting in a new battery and click the shutter button to reset the mirror. It just might work.

Alright that's it for now. Thanks for reading. Hope this helps out some peeps.

Harmony Touch Touchscreen Not Responding

This is really one of those 'may as well try' solutions that will really only help lazy people like myself. So my dad got me a brand new Harmony Touch remote - not the big square slate, but the one that they just released in 2012 that's more like a regular Harmony remote with a touch screen in the middle:

http://www.logitech.com/en-us/product/harmony-touch

So I brought it home, opened it and it looked really nice. I was able to hook it up, setup a new myharmony.com account, import my old harmony one settings - and program the remote. All the physical buttons worked out fine too - but the touch screen itself was crazy slow, buggy, and sometimes completely unresponsive. I tried resetting the remote, powering it on and off - but still the touch screen was horrid.

I logged a support ticket with logitech - but decided - rather than, wait - at least I should try exchanging it. So I exchanged the remote for another one - and huzzah - the touchscreen was fine. So it looked like a manufacturing defect. Rather than waste your time searching forums, trying firmware updates, and resets - try the easiest thing first and exchange your remote.

Again - not brain surgery - but lazy people like me hate dealing with the hassles of returns and exchanges. So hopefully this helps some peeps.

Asus RT-N66U, Asus EA-N66, and Airplay

Alright, long story short, here was my setup:

Asus RT-N66U as my main wireless Access Point. Asus EA-N66 acting in Repeater Mode. And NO Airplay was working. No matter what I connected my devices to - the main Access Point, or the Repeater, Airplay would not work. Airplay devices would not show up. Just basically at a loss.

After trying a bunch of things - I e-mailed ASUS - and as it seems with most routers - the issue was the firmware.

I upgraded the EA-N66 Repeater's firmware to the latest at this link:

http://support.asus.com.cn/Download.aspx?SLanguage=en&p=11&s=1&m=EA-N66&os=30&ft=20&f_name=FW_EA_N66_1017d.zip#FW_EA_N66_1017d.zip

And all was fine! Everything was working again!

Hope this helps some peeps.

jQuery animate scrollTop Flicker

Hey everybody,

was doing some scrolling animations on my one-page site using jQuery's animate function.

This is the basics of what I had:


$(function(){
  $("#link").click(function(){ 
    navigateTo(".section");
  });
});

function navigateTo(destination) {
  $('html,body').animate({scrollTop: $(destination).offset().top - 48},'slow');
}

The issue I was seeing is that the animate was happening, but so was a flicker. The flicker was caused by the default behaviour of clicking on a link with href="#". The problem is the default behvaiour of that is to scroll to the top. So it would quickly scroll to the top, then scroll to the destination. Solution? Block the default behaviour by returning false in the event handler of the click.


$(function(){
  $("#link").click(function(){ 
    navigateTo(".section");
    return false;
  });
});

function navigateTo(destination) {
  $('html,body').animate({scrollTop: $(destination).offset().top - 48},'slow');
}

Hopefully that will help some peeps.

Nest Thermostat Heating Issues

Problem

Alright, so I absolutely love the Nest - as it seems every Nest owner does. I was using it all leisurely during the summer and enjoying all of its benefits - but then when winter came - my Nest starting wigging out.

I thought originally it might have been my setup at my condo. My condo is one of those buildings where it's either "COOL" or "HEAT" on. Not both. So at one point in the winter, the building switched the heat on.

Now, even with my old thermostat - it's one of those on/off dealies. So if my thermostat was set to cool - the heat would actually turn on when the thermostat was "Cooling". But that's a separate issue. This is true of all thermostats for these types of buildings.

No - the issue that I was having separate. When I switched to heating, on the Nest - and when I set it to temperature that was higher than the current room temperature - the Nest should have started heating my place right?

Wrong.

Here's what happened (ie. the symptoms in case you're having the same issue)

Setup

• Nest V1
• Electric Forced Air Furnace
• Wires: RH, Y1, W1, G

Symptons

• The Nest was in "HEAT" mode
• I set the Nest to a temperature hotter than the current temperature to engage heating
• The display would show "Heating" for about 2 seconds, and then go to "Delayed for 3:00 Minutes". My fan would kick on for a bit, but that was it.
• After the 3:00 minutes, same thing would happen "Heating" for about 2 seconds, and then back into "Delayed for 3:00 Minutes"
• Here's the telltale sign - just after "Heating" for about 2 seconds, and just before "Delayed for 3:00 Minutes", the little green LED in the sensor area would blink (this should have been a sign to me).
• So it would end up in the endless loop of "Heating" for about 2 seconds, and "Delayed for 3:00 Minutes" with that green LED flashing.

I called Nest Support - and it turns out it was a Voltage issue. Even though I was getting good voltage readings, there was an issue.

How to solve it?

I'm not taking any liability for this - and you should probably call Nest support - but you can try the follwing.

Temporary Work-Around

This has actually become my permanent work-around, because I don't feel like re-wiring my condo. The work-around is to UNPLUG THE COOLING "Y1" WIRE from the Nest during Heating seasons like Winter - so you just have RH, W1, and G. This apparently reduces the load or something. This works perfectly for me. I just have to unplug the wire/plug it back in once a year.

Permanent Fix

Nest support told me to run a wire from the furnace board's "Common" terminal to the "C" terminal on the Nest. Basically you need a common wire. However, I didn't have any extra wires long enough in my setup to do this.

Hope this helps some peeps.