Publishing a Django Project to Heroku using Git from Windows 8.1

Problem

I wanted to deploy my Django project to Heroku from Windows, but never done it before. My particular circumstances are:

• This is the first time I try deploying to Heroku from this PC.
• I don't know how to deploy a Django project to Heroku
• I have installed Git for Windows
• I was developing my Django project in Visual Studio 2012 using Python Tools for Visual Studio (I will explain this in the next section)
• I have not setup any SSH private/public keys on this PC

I will not cover:

• How to setup an account with Heroku
• How to install Heroku, Git, Python, Visual Studio, PTVS on Windows
• How to install, create a new project, and program in Django
• How to use Git.

For Git, I will not go into details so you need to at least know the basics of using git (git add, git commit, git remote, git push). I will explain my experience, and the steps that worked for me getting my project up and running in Heroku. I'm writing this as a reminder for myself on how to do these things, but hopefully this will be helpful to someone out there in the same situation.

Side note: I will interject every now and then with side notes as I explain things, just because that's how my thought process works, or it's just difficult to put it in a logical step. Sorry in advance.

Django Development in VS2012 using PTVS

Python Tools for Visual Studio is a great plugin for Visual Studio 2012 to program Python [website].  I started my Django project in VS2012 using PTVS because it provides the ability to create a new Django project by default. Additionally, VS2012 provides a pretty good interface with Git if you want to manage your program/project with Git (requires a separate installation of Git plus a VS plugin for it).

Unfortunately, this feature does not work well if you want to publish your Django project on Heroku.  You see, VS only allows you to add the whole VS project (a.k.a. "solution" in VS terminology) to source control. When creating a new project in VS, there is a *.sln file that gets created as your 'VS project' designator. When you create a Django project in VS, this resides one level above your Django project's root folder ("VS project").  Meanwhile, if you are going to upload your project to Heroku using Git, Heroku requires that you are uploading the root directory of your project. In other words, your git repository starts at the Django project's root folder ("Django root").

Conclusion: can't upload our Django project to Heroku from VS2012 - until they allow 'Git init' at Django root.

Aside from that, developing Python in VS2012 using PTVS is actually pretty good. You should at least give it a try.

Versions etc.

The pieces of software I used are:

• Git version 1.8.1.msysgit.1
• Python 3.3, with modules:
• Django 1.6.1
• distribute 0.7.3
• gunicorn 18.0
• virtualenv 1.10.1
• Windows 8.1
• PuTTY, PuTTYgen, Pageant beta 0.63
• heroku 3.2.1 (I think ...)

Heroku

Heroku provides a pretty good Getting-Started-With-Django tutorial on their website [link]. That tutorial will help you set up your Heroku account and creating a new Heroku project. Also, it covers 80% what you need to upload your Django project to Heroku. The missing 20% is how to do it in Windows.

Here, I will write down some missing things they don't mention in case of Windows.

Git

Well, now that I found out I can't upload my project through VS although it's already maintained using Git, I have to make a new git repo that starts at my Django project's root folder. 

1. Copied my Django root project to a different location (e.g. C:\Projects\DjangoProject1).
2. Start a git repo there.

Oops - side note: USE Git Bash. Git Bash is a terminal-like tool that you get by installing Git for Windows (imagine that?). If you don't know how to find it, just hit the Windows button and type in "Git bash".

In Git bash:

~/Projects$ cd DjangoProject1

~/Projects/DjangoProject1$ git init

That will create a new git repository for DjangoProject1. Before you add/commit add an entry in .gitignore to exclude the virtual environment folder. Create a new .gitignore file in the root of your git repo (I like to use vim)

~/Projects/DjangoProject1$ vim .gitignore

So if my virtual environment directory is in  /Projects/DjangoProject1/env, in .gitignore, type:

env/

Save the file and close the editor (:wq in vim).

Side note: ALWAYS use a virtual environment whenever you can. For Python (and hence, Django) it's virtualenv. If you're not familiar on how to do this, here's some links to some good tutorials [link].

Next, add your project to the git repo:

~/Projects/DjangoProject1$ git add -A

~/Projects/DjangoProject1$ git commit -m "First commit"

So this is your master branch. 

Next, add the Heroku git path as your remote. If you followed Heroku's guide and run "heroku create" (from this step) from your repo, that path is automatically added to your git remote.  Check it using 

~/Projects/DjangoProject1$ git config -l (lower case L)

You should see a remote.heroku.url= entry at the bottom of the config.

According to Heroku's guide, your next step would be doing:

~/Projects/DjangoProject1$ git push heroku master

Which should push your master branch to heroku. However, when I did it then, it gave me a "Permission denied" error.  Turns out, I need to create and add SSH keys for my system.

SSH Keys

Unfortunately Windows does not come with tools to generate SSH keys by default. I needed to download PUTTYgen to generate my SSH keys. I followed this tutorial. It's pretty straightforward, except at step 6.

Step 6 in that tutorial said to save the private key. By default, PUTTYgen will ask you to give a file name and it will have .ppk extension. To make this SSH keys work for my Windows machine, I had to do several things not mentioned in that tutorial:

1. Create a .ssh directory under C:\Users\MyUserName. Like this:
   
   C:\Users\MyUserName\.ssh
   
   Note: if you can't create the folder from Windows Explorer, use Git Bash (or Window's Command Prompt) and use 'mkdir' command:
   
   Git bash: ~/Users/MyUserName$ mkdir .ssh
   Command Prompt: C:\Users\MyUserName > mkdir .ssh
2. Once the keys are generated from that tutorial, save the private key using (from the toolbar) Conversions > Export OpenSSH keys
3. Save the private key with file name "id_rsa" in C:\Users\MyUserName\.ssh
4. Save the public key with file name "id_rsa.pub" in C:\Users\MyUserName\.ssh
5. Add the keys to Heroku (from this guide)
   
   ~/Projects/DjangoProject1$ heroku keys:add

Now here's the confusing part for me. Before that, I tried different solutions including:

• Installed Pageant and adding the keys to it
• copied the private and public keys to C:\Program Files (x86)\Git\.ssh (private key not exported as OpenSSH keys at this point)

Which the upload kept failing. My upload to Heroku started working AFTER I save the private key using step 2 above (I think I might've copied the OpenSSH version of the private key to C:\Program Files (x86)\Git\.ssh as well ... I can't remember). So, I'm not sure whether or not saving the keys in Pageant and Git\.ssh have anything to do with it or not.  But now it's working. WAIT! There are two more things to do before Heroku actually accepts and recognize the project as a Django project: creating a requirements.txt file and Procfile file.

Side note: without the requirements.txt, the upload will fail with this error.

Pip freeze > requirements.txt

I have to create a requirements.txt file to let Heroku know what modules I need for my Django project. The requirements.txt file must reside in the Django project's root directory. To do this:

1. Activate the virtual environment if it's not already. If not, all the globally-installed Python modules will be included, and we don't want that (this is the other important benefit of using virtual environments)
   
   ~/Projects/DjangoProject1$ source env/bin/activate
   (env)~/Projects/DjangoProject1$
   
   The (env) indicates that the virtual environment has been activated.
2. Run the pip freeze command:
   
   (env)~/Projects/DjangoProject1$ pip freeze > requirements.txt
   
   This will create the requirements.txt file in the root directory.

For my simple project, all I'm using is Django, distribute, and gunicorn. Here's the content of my requirements.txt file:

requirements.txt

Gunicorn is required (as you can see from this section of the Heroku tutorial) - your app won't launch without it even if you successfully upload it to Heroku.

Procfile

Create a Procfile (capital P, lowercase f, no extensions) in the Django project root directory as explained in the Heroku tutorial section Declare process types with Procfile. Here's the content of my Procfile file:

Start Uploading!

Here's the checklist for uploading:

• Git repo root folder is Django project root folder (all project files except the virtual environment files have been added to the repo)
• Have the Heroku git repo as remote
• gunicorn module is installed
• requirements.txt and Procfile are in the Django project root folder
• SSH keys saved as OpenSSH and saved in C:\Users\MyUserName\.ssh

Upload to Heroku using:

(env)~/Projects/DjangoProject1$ git push heroku master

That command basically push the master branch to heroku (remote repo).

When the upload is successful, I get something like this:

Additionally, if everything works and running, Heroku can launch my application automatically. I can check that it's running using "heroku ps" like so:

(I'm hiding the actual name of my app, sorry)

So when I go to my app url http://pacific-ravine-8494.herokuapp.com/ it's running! Yay!

References:

Heroku Getting Started with Django: https://devcenter.heroku.com/articles/getting-started-with-django

Heroku Git tutorial: https://devcenter.heroku.com/articles/git

Manually Generating SSH Keys in Windows: http://wiki.joyent.com/wiki/display/jpc2/Manually+Generating+Your+SSH+Key+in+Windows

Create, Connect and Publish using Visual Studio with Git: http://blogs.msdn.com/b/visualstudioalm/archive/2013/02/06/set-up-connect-and-publish-using-visual-studio-with-git.aspx

How to use Visual Studio 2012 and BitBucket: http://ilblogdipego.blogspot.com/2013/03/how-to-use-git-with-visual-studio-2012.html

Trouble with Windows and SSH Keys: http://www.gmarwaha.com/blog/2011/05/18/heroku-trouble-with-windows-and-ssh-keys/

How to Install SSH private key generated by Puttygen (Ubuntu): http://askubuntu.com/questions/15378/how-do-i-install-a-ssh-private-key-generated-by-puttygen 

Git push heroku master rejected: http://stackoverflow.com/questions/14939881/git-push-heroku-master-heroku-push-rejected-no-cedar-supported-app-detected

.gitignore not ignoring: http://stackoverflow.com/questions/1139762/gitignore-file-not-ignoring

virtualenv tutorials for beginners: http://stackoverflow.com/questions/5844869/comprehensive-beginners-virtualenv-tutorial

Ubuntu 12.04 guest OS runs in "Low Graphics Mode" on VirtualBox 4.2.14

As the title says, this entry is about the persistent issue with running Ubuntu 12.04 as guest OS on VirtualBox 4.2.14 running in "Low Graphics Mode" - meaning it won't let you boot into Ubuntu with its nice graphical user interface, instead you're stuck with the shell/terminal. If you google (yes, that's officially a word) this issue, you'll find some discussions of it like in this forum: VirtualBox Forum. Before I move on, a couple notes which may or may not be important:

1. I used Ubuntu 12.04 Desktop edition 32-bit version
2. I'm writing this based on my recent experience with the issue where my Host OS is Windows 7 (SP1) 64-bit. But I think it happens regardless of the kind of Host OS (MacOSX, Windows...)
3. I tried increasing my video memory to 48MB but it does not solve the issue

The only known solution for this problem so far is installing the VirtualBox Guest Addition package. Once you get the Guest Addition installed, the problem goes away. However, I did run into some problems trying to install the Guest Additions. Hence, I am writing this note to help any of you who might encounter similar problems (but also for my own notes). I apologize in advance for not having screenshots that would greatly help the explanations below. I'll try to reproduce the issue and update this post with screenshots as soon as I can. So here goes...

I'm going to skip the steps of installing VirtualBox and the Ubuntu as guest OS. I'm going to assume you are at the point where you've installed those two, and as you load your Ubuntu VM, you're seeing this issue.

 

Mounting the Guest Additions .iso

The scenario:

• I've launched my Ubuntu VM
• It gives me a message "Running in Low Graphics Mode"
• I choose to "Continue running in Low Graphics Mode"
• The VM launches into shell, I logged in
• Now I'm in the shell prompt

Now I'm trying to mount the Guest Additions .iso to install it.  I do:

1. On the VM menu bar, go to: Devices > CD/DVD Devices
2. Choose: VBoxGuestAdditions.iso
3. I noticed I don't have a checkbox next to it after I selected it, but there's no error messages.

I tried looking around for the mount point for the VBoxGuestAdditions.iso, under: /media and /mnt by typing the following in the shell:

• ~$ ls /media
• ~$ ls /mnt

 And both directories remained empty.  This means the mount somehow didn't work. I tried to unmount/re-mount this a few times, even manually pointing to the .iso file (in Windows, it's under: C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso), nothing works.

Solution:

1. Download the Guest Additions .iso for VirtualBox 4.2.14 from the archive: here.  
2. Download and save the .iso somewhere (maybe your default \Downloads folder).
3. Mount the .iso by:
1. On the VM menu bar, go to: Devices > CD/DVD Devices
2. Click on "Choose a virtual CD/DVD disk file..."
3. Find your VBoxGuestAdditions_4.2.14.iso file (do you remember where you saved it?), click on the file, and click Open.
4. If all goes well, now when you reopen Devices > CD/DVD Devices menu, you'll see a checkbox next to "VBoxGuestAdditions_4.2.14.iso"

Wait, we're not done yet.  After I did this, when I ls on the /media and /mnt folders again, I still don't see anything mounted. I guess I have to mount it manually.

Mounting:

1. In the Ubuntu shell, do:
• ~$ ls /dev OR ~$ ls /dev/sr0
2. If sr0 exists, then you're in luck! That's what you need to mount. Otherwise, I have no answer.
3. Create a directory as the mount point for your VBoxGuestAdditions_4.2.14.iso:
1. ~$ mkdir /media/disk1
• Note: I don't remember if you have to use sudo mkdir /media... or not. If you get permission error then you probably do. "/disk1" is a name I arbitrarily chose, so you can use any name you want if you so choose.
4. Edit the fstab file to let the system know you're adding a mount point.
1. ~$ sudo vi /etc/fstab 

• Make sure to use sudo to get permission to write the file
• I use vi, but you can use any other text editor you like, e.g. pico)
2. Add a new line to the file (the last line in the screenshot):
   /dev/sr0     /media/disk1   auto   ro,noauto,user,exec 0  0
3. I'm not sure how important the exact spacings between those entries are, but  I tried to make the first character line up with the first character of the column name when I can.
4. Save the file and close the text editor.
5. Mount the .iso using:
1. ~$ sudo mount /dev/sr0 /media/disk1
• Yes, you have to use sudo due to permission requirement
6. If all goes well, now you can go to the directory and execute the script to install the Guest Additions:
   
1. ~$ cd /media/disk1
2. ~$ sudo sh VBoxLinuxAdditions.run

The Guest Additions should install. Reboot after it's finished installing, and the "Low Graphics Mode" message should not appear anymore and take you to Ubuntu login screen.

Hope that helps.