Publishing a Django Project to Heroku using Git from Windows 8.1

Problem

I wanted to deploy my Django project to Heroku from Windows, but never done it before. My particular circumstances are:

• This is the first time I try deploying to Heroku from this PC.
• I don't know how to deploy a Django project to Heroku
• I have installed Git for Windows
• I was developing my Django project in Visual Studio 2012 using Python Tools for Visual Studio (I will explain this in the next section)
• I have not setup any SSH private/public keys on this PC

I will not cover:

• How to setup an account with Heroku
• How to install Heroku, Git, Python, Visual Studio, PTVS on Windows
• How to install, create a new project, and program in Django
• How to use Git.

For Git, I will not go into details so you need to at least know the basics of using git (git add, git commit, git remote, git push). I will explain my experience, and the steps that worked for me getting my project up and running in Heroku. I'm writing this as a reminder for myself on how to do these things, but hopefully this will be helpful to someone out there in the same situation.

Side note: I will interject every now and then with side notes as I explain things, just because that's how my thought process works, or it's just difficult to put it in a logical step. Sorry in advance.

Django Development in VS2012 using PTVS

Python Tools for Visual Studio is a great plugin for Visual Studio 2012 to program Python [website].  I started my Django project in VS2012 using PTVS because it provides the ability to create a new Django project by default. Additionally, VS2012 provides a pretty good interface with Git if you want to manage your program/project with Git (requires a separate installation of Git plus a VS plugin for it).

Unfortunately, this feature does not work well if you want to publish your Django project on Heroku.  You see, VS only allows you to add the whole VS project (a.k.a. "solution" in VS terminology) to source control. When creating a new project in VS, there is a *.sln file that gets created as your 'VS project' designator. When you create a Django project in VS, this resides one level above your Django project's root folder ("VS project").  Meanwhile, if you are going to upload your project to Heroku using Git, Heroku requires that you are uploading the root directory of your project. In other words, your git repository starts at the Django project's root folder ("Django root").

Conclusion: can't upload our Django project to Heroku from VS2012 - until they allow 'Git init' at Django root.

Aside from that, developing Python in VS2012 using PTVS is actually pretty good. You should at least give it a try.

Versions etc.

The pieces of software I used are:

• Git version 1.8.1.msysgit.1
• Python 3.3, with modules:
• Django 1.6.1
• distribute 0.7.3
• gunicorn 18.0
• virtualenv 1.10.1
• Windows 8.1
• PuTTY, PuTTYgen, Pageant beta 0.63
• heroku 3.2.1 (I think ...)

Heroku

Heroku provides a pretty good Getting-Started-With-Django tutorial on their website [link]. That tutorial will help you set up your Heroku account and creating a new Heroku project. Also, it covers 80% what you need to upload your Django project to Heroku. The missing 20% is how to do it in Windows.

Here, I will write down some missing things they don't mention in case of Windows.

Git

Well, now that I found out I can't upload my project through VS although it's already maintained using Git, I have to make a new git repo that starts at my Django project's root folder. 

1. Copied my Django root project to a different location (e.g. C:\Projects\DjangoProject1).
2. Start a git repo there.

Oops - side note: USE Git Bash. Git Bash is a terminal-like tool that you get by installing Git for Windows (imagine that?). If you don't know how to find it, just hit the Windows button and type in "Git bash".

In Git bash:

~/Projects$ cd DjangoProject1

~/Projects/DjangoProject1$ git init

That will create a new git repository for DjangoProject1. Before you add/commit add an entry in .gitignore to exclude the virtual environment folder. Create a new .gitignore file in the root of your git repo (I like to use vim)

~/Projects/DjangoProject1$ vim .gitignore

So if my virtual environment directory is in  /Projects/DjangoProject1/env, in .gitignore, type:

env/

Save the file and close the editor (:wq in vim).

Side note: ALWAYS use a virtual environment whenever you can. For Python (and hence, Django) it's virtualenv. If you're not familiar on how to do this, here's some links to some good tutorials [link].

Next, add your project to the git repo:

~/Projects/DjangoProject1$ git add -A

~/Projects/DjangoProject1$ git commit -m "First commit"

So this is your master branch. 

Next, add the Heroku git path as your remote. If you followed Heroku's guide and run "heroku create" (from this step) from your repo, that path is automatically added to your git remote.  Check it using 

~/Projects/DjangoProject1$ git config -l (lower case L)

You should see a remote.heroku.url= entry at the bottom of the config.

According to Heroku's guide, your next step would be doing:

~/Projects/DjangoProject1$ git push heroku master

Which should push your master branch to heroku. However, when I did it then, it gave me a "Permission denied" error.  Turns out, I need to create and add SSH keys for my system.

SSH Keys

Unfortunately Windows does not come with tools to generate SSH keys by default. I needed to download PUTTYgen to generate my SSH keys. I followed this tutorial. It's pretty straightforward, except at step 6.

Step 6 in that tutorial said to save the private key. By default, PUTTYgen will ask you to give a file name and it will have .ppk extension. To make this SSH keys work for my Windows machine, I had to do several things not mentioned in that tutorial:

1. Create a .ssh directory under C:\Users\MyUserName. Like this:
   
   C:\Users\MyUserName\.ssh
   
   Note: if you can't create the folder from Windows Explorer, use Git Bash (or Window's Command Prompt) and use 'mkdir' command:
   
   Git bash: ~/Users/MyUserName$ mkdir .ssh
   Command Prompt: C:\Users\MyUserName > mkdir .ssh
2. Once the keys are generated from that tutorial, save the private key using (from the toolbar) Conversions > Export OpenSSH keys
3. Save the private key with file name "id_rsa" in C:\Users\MyUserName\.ssh
4. Save the public key with file name "id_rsa.pub" in C:\Users\MyUserName\.ssh
5. Add the keys to Heroku (from this guide)
   
   ~/Projects/DjangoProject1$ heroku keys:add

Now here's the confusing part for me. Before that, I tried different solutions including:

• Installed Pageant and adding the keys to it
• copied the private and public keys to C:\Program Files (x86)\Git\.ssh (private key not exported as OpenSSH keys at this point)

Which the upload kept failing. My upload to Heroku started working AFTER I save the private key using step 2 above (I think I might've copied the OpenSSH version of the private key to C:\Program Files (x86)\Git\.ssh as well ... I can't remember). So, I'm not sure whether or not saving the keys in Pageant and Git\.ssh have anything to do with it or not.  But now it's working. WAIT! There are two more things to do before Heroku actually accepts and recognize the project as a Django project: creating a requirements.txt file and Procfile file.

Side note: without the requirements.txt, the upload will fail with this error.

Pip freeze > requirements.txt

I have to create a requirements.txt file to let Heroku know what modules I need for my Django project. The requirements.txt file must reside in the Django project's root directory. To do this:

1. Activate the virtual environment if it's not already. If not, all the globally-installed Python modules will be included, and we don't want that (this is the other important benefit of using virtual environments)
   
   ~/Projects/DjangoProject1$ source env/bin/activate
   (env)~/Projects/DjangoProject1$
   
   The (env) indicates that the virtual environment has been activated.
2. Run the pip freeze command:
   
   (env)~/Projects/DjangoProject1$ pip freeze > requirements.txt
   
   This will create the requirements.txt file in the root directory.

For my simple project, all I'm using is Django, distribute, and gunicorn. Here's the content of my requirements.txt file:

requirements.txt

Gunicorn is required (as you can see from this section of the Heroku tutorial) - your app won't launch without it even if you successfully upload it to Heroku.

Procfile

Create a Procfile (capital P, lowercase f, no extensions) in the Django project root directory as explained in the Heroku tutorial section Declare process types with Procfile. Here's the content of my Procfile file:

Start Uploading!

Here's the checklist for uploading:

• Git repo root folder is Django project root folder (all project files except the virtual environment files have been added to the repo)
• Have the Heroku git repo as remote
• gunicorn module is installed
• requirements.txt and Procfile are in the Django project root folder
• SSH keys saved as OpenSSH and saved in C:\Users\MyUserName\.ssh

Upload to Heroku using:

(env)~/Projects/DjangoProject1$ git push heroku master

That command basically push the master branch to heroku (remote repo).

When the upload is successful, I get something like this:

Additionally, if everything works and running, Heroku can launch my application automatically. I can check that it's running using "heroku ps" like so:

(I'm hiding the actual name of my app, sorry)

So when I go to my app url http://pacific-ravine-8494.herokuapp.com/ it's running! Yay!

References:

Heroku Getting Started with Django: https://devcenter.heroku.com/articles/getting-started-with-django

Heroku Git tutorial: https://devcenter.heroku.com/articles/git

Manually Generating SSH Keys in Windows: http://wiki.joyent.com/wiki/display/jpc2/Manually+Generating+Your+SSH+Key+in+Windows

Create, Connect and Publish using Visual Studio with Git: http://blogs.msdn.com/b/visualstudioalm/archive/2013/02/06/set-up-connect-and-publish-using-visual-studio-with-git.aspx

How to use Visual Studio 2012 and BitBucket: http://ilblogdipego.blogspot.com/2013/03/how-to-use-git-with-visual-studio-2012.html

Trouble with Windows and SSH Keys: http://www.gmarwaha.com/blog/2011/05/18/heroku-trouble-with-windows-and-ssh-keys/

How to Install SSH private key generated by Puttygen (Ubuntu): http://askubuntu.com/questions/15378/how-do-i-install-a-ssh-private-key-generated-by-puttygen 

Git push heroku master rejected: http://stackoverflow.com/questions/14939881/git-push-heroku-master-heroku-push-rejected-no-cedar-supported-app-detected

.gitignore not ignoring: http://stackoverflow.com/questions/1139762/gitignore-file-not-ignoring

virtualenv tutorials for beginners: http://stackoverflow.com/questions/5844869/comprehensive-beginners-virtualenv-tutorial

Ubuntu 12.04 guest OS runs in "Low Graphics Mode" on VirtualBox 4.2.14

As the title says, this entry is about the persistent issue with running Ubuntu 12.04 as guest OS on VirtualBox 4.2.14 running in "Low Graphics Mode" - meaning it won't let you boot into Ubuntu with its nice graphical user interface, instead you're stuck with the shell/terminal. If you google (yes, that's officially a word) this issue, you'll find some discussions of it like in this forum: VirtualBox Forum. Before I move on, a couple notes which may or may not be important:

1. I used Ubuntu 12.04 Desktop edition 32-bit version
2. I'm writing this based on my recent experience with the issue where my Host OS is Windows 7 (SP1) 64-bit. But I think it happens regardless of the kind of Host OS (MacOSX, Windows...)
3. I tried increasing my video memory to 48MB but it does not solve the issue

The only known solution for this problem so far is installing the VirtualBox Guest Addition package. Once you get the Guest Addition installed, the problem goes away. However, I did run into some problems trying to install the Guest Additions. Hence, I am writing this note to help any of you who might encounter similar problems (but also for my own notes). I apologize in advance for not having screenshots that would greatly help the explanations below. I'll try to reproduce the issue and update this post with screenshots as soon as I can. So here goes...

I'm going to skip the steps of installing VirtualBox and the Ubuntu as guest OS. I'm going to assume you are at the point where you've installed those two, and as you load your Ubuntu VM, you're seeing this issue.

 

Mounting the Guest Additions .iso

The scenario:

• I've launched my Ubuntu VM
• It gives me a message "Running in Low Graphics Mode"
• I choose to "Continue running in Low Graphics Mode"
• The VM launches into shell, I logged in
• Now I'm in the shell prompt

Now I'm trying to mount the Guest Additions .iso to install it.  I do:

1. On the VM menu bar, go to: Devices > CD/DVD Devices
2. Choose: VBoxGuestAdditions.iso
3. I noticed I don't have a checkbox next to it after I selected it, but there's no error messages.

I tried looking around for the mount point for the VBoxGuestAdditions.iso, under: /media and /mnt by typing the following in the shell:

• ~$ ls /media
• ~$ ls /mnt

 And both directories remained empty.  This means the mount somehow didn't work. I tried to unmount/re-mount this a few times, even manually pointing to the .iso file (in Windows, it's under: C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso), nothing works.

Solution:

1. Download the Guest Additions .iso for VirtualBox 4.2.14 from the archive: here.  
2. Download and save the .iso somewhere (maybe your default \Downloads folder).
3. Mount the .iso by:
1. On the VM menu bar, go to: Devices > CD/DVD Devices
2. Click on "Choose a virtual CD/DVD disk file..."
3. Find your VBoxGuestAdditions_4.2.14.iso file (do you remember where you saved it?), click on the file, and click Open.
4. If all goes well, now when you reopen Devices > CD/DVD Devices menu, you'll see a checkbox next to "VBoxGuestAdditions_4.2.14.iso"

Wait, we're not done yet.  After I did this, when I ls on the /media and /mnt folders again, I still don't see anything mounted. I guess I have to mount it manually.

Mounting:

1. In the Ubuntu shell, do:
• ~$ ls /dev OR ~$ ls /dev/sr0
2. If sr0 exists, then you're in luck! That's what you need to mount. Otherwise, I have no answer.
3. Create a directory as the mount point for your VBoxGuestAdditions_4.2.14.iso:
1. ~$ mkdir /media/disk1
• Note: I don't remember if you have to use sudo mkdir /media... or not. If you get permission error then you probably do. "/disk1" is a name I arbitrarily chose, so you can use any name you want if you so choose.
4. Edit the fstab file to let the system know you're adding a mount point.
1. ~$ sudo vi /etc/fstab 

• Make sure to use sudo to get permission to write the file
• I use vi, but you can use any other text editor you like, e.g. pico)
2. Add a new line to the file (the last line in the screenshot):
   /dev/sr0     /media/disk1   auto   ro,noauto,user,exec 0  0
3. I'm not sure how important the exact spacings between those entries are, but  I tried to make the first character line up with the first character of the column name when I can.
4. Save the file and close the text editor.
5. Mount the .iso using:
1. ~$ sudo mount /dev/sr0 /media/disk1
• Yes, you have to use sudo due to permission requirement
6. If all goes well, now you can go to the directory and execute the script to install the Guest Additions:
   
1. ~$ cd /media/disk1
2. ~$ sudo sh VBoxLinuxAdditions.run

The Guest Additions should install. Reboot after it's finished installing, and the "Low Graphics Mode" message should not appear anymore and take you to Ubuntu login screen.

Hope that helps.

MacPorts trouble with OS X Lion

It's a llama and an ass.

Recently I had trouble using MacPorts on Lion, for whatever reason. But I managed to find the fix, and here they are.

1. Unable to do selfupdate:

   
   ~$ sudo port selfupdate
   ---> Updating MacPorts base sources using rsync
   MacPorts base version 2.0.4 installed,
   MacPorts base version 2.1.2 downloaded.
   ---> Updating the ports tree
   ---> MacPorts base is outdated, installing new version 2.1.2
   Installing new MacPorts release in /opt/local as root:admin; permissions 0755; Tcl-Package in /Library/Tcl
   
   Error: /opt/local/bin/port: port selfupdate failed: Error installing new MacPorts base: shell command failed (see log for details)
   
   Cause: ??? (shame on me for not checking the logs)
   Solution: Re-install Xcode (ref: MacPorts ticket #30507).
   Before re-installing Xcode, I had to uninstall my existing one. All I did was:
   1. Search for the Xcode executable from Spotlight
   2. Open the folder where the Xcode executable is (usually under /Applications).
   3. Drag the icon to the Trash bin. OR drag the Launcher icon to the Trash bin (this will prompt for are-you-sure-you-want-to-uninstall? confirmation). Click Yes/OK if asked.
   Unfortunately, I wasn't 100% sure my method of uninstalling Xcode is correct. So here's a most-likely-more-proper-way reference from StackOverflow: #6777462 - Can't Reinstall Xcode
   If you're able to successfully re-install XCode (mine was 4.4.1 as of this writing), you should be able to run ~$ sudo port selfupdate successfully.
   
   p.s. Actually, I also installed MacPorts 2.1.2 manually (go to MacPorts website, download the .dmg, install, etc.). So now I'm not entirely sure which fixed it.

2. Unable to install Octave

   
   ~$ sudo port install octave
   Warning: The Command Line Tools for Xcode don't appear to be installed; most ports will likely fail to build.
   Warning: See http://guide.macports.org/chunked/installing.xcode.html for more information.
   Error: Unable to open port: can't read "build.cmd": Failed to locate 'make' in path: '/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin' or at its MacPorts configuration time location, did you move it?
   Error: Unable to execute port: upgrade perl5 failed
   
   Cause: (the error message was pretty explanatory on this one) "The Command Line Tools for Xcode don't appear to be installed" and (hence) "Failed to locate 'make' in path".
   Solution: Install the XCode Command Line Tool.
   
   1. Launch XCode.
   2. In XCode, go to: Preferences > Downloads
   3. Click "Install" for the "Command Line Tools" item (the screenshot shows that it's already installed).
      
      
      
      
      
   Currently I'm still unable to install octave because MacPorts failed to install gnuplot, but it's not the error above - so that's an improvement. It may be missing some dependencies...
   
   p.s. Prior to installing the Command Line Tools, I ran across this post: "GCC non-existent in Lion". So I also did what was recommended there: made a symbolic link to llvm-gcc-4.2 to /usr/bin/gcc. But the location of the newest Xcode directory in my system is a little different:
   
   ~$ sudo ln -s /Applications/Xcode.app/Contents/Developer/usr/llvm-gcc-4.2/bin/llvm-gcc-4.2 /usr/bin/gcc

VirtualBox and The Pesky Shared Folder

VirtualBox and Shared Folders

What I learned: if you want full access to your shared folder on VirtualBox, do manual mounts.


System:
Host: Mac OSX 10.7 (a.k.a. "Lion")
Guest: Ubuntu 11.10 (a.k.a. "Oneiric Ocelot")
VirtualBox version: 4.1.10 r76795
VirtualBox Guest Addition version: 4.1.10
Shared Folder name: ~/sharedfolder

Case:
I need to rebuild a database in the guest OS. The database dump is stored in the shared folder (due to limited space in my .vdi), and I need to use it to rebuild the database.

Issue:
I used the command:
~$ sudo su postgres -c "psql -d < path-to-shared-folder/database.pgdump"

Result:
after entering root password, it returned:
"Permission denied"

I checked the permission on the shared folder and the dump file. The file's permission was:
-rwxr-x---

I tried changing the permission to no avail.

Solution: manual mount!

1. Create a new folder in your HOST system. e.g. ~/sharedfolder
2. Add the folder to your VirtualBox Guest OS shared folder list
1. To access the Shared Folders option: select your guest OS > Settings > Shared Folders
   
2. Add your shared folder. Make sure NOT to check "Auto-mount" (it's evil). Then click "OK".
   
3. Run your guest OS. The following instructions are for Ubuntu only (sorry!).
4. In your guest OS, create the directory you want the shared folder to be mounted to. For example: /mnt/shared
   
   
5. Open a Terminal and mount your shared folder
   
   
6. Et voila, your file (if you have any in the shared folder) will have the following permission:
   -rwxr-xr-x 1 root root

Only after doing the manual mount, I was able to use the .pgdump file to rebuild the database. Prior to that, it's just a bunch of "Permission denied", "Wrong protocol", "file or folder not found", etc. I can't say whether or not this would work if you want to use a shared folder you already used with Auto-mount. I had to create a new folder for it to work. Let me know if you had a different experience/luck. Signing off...

iPadHD = iP+ADHD

The new iPad will be called iPadHD:

i - P - Attention Deficit Hyperactivity Disorder (ADHD)

What is ADHD?

OS X 10.7 (Lion) Tips - Adding a Folder to Finder Sidebar/Favorites

1. In Finder, select the folder you want to add
2. ⌘ (command) + T

VirtualBox Shared Folder (Mac OS X 10.6)

I believe it's generally a good idea to set up a shared folder if you're doing virtual machines (VM's) like VirtualBox.  Particularly:

• If you have limited disk space (like me), you'd want to make your virtual disk fixed-sized. But at the same time, you recognize you may do a lot of work with your VM, so you want some way to store your data without quickly hitting the capacity of your virtual disk.
• You may have some files you'll be working on both in your VM OS, and your host/main OS. So you need a way to transfer/access the files between the two OS'es.

Here are the steps to set up your shared folder for VirtualBox (version 4.1.4 as of this writing) with Mac OS X 10.6 as host, with Xubuntu 11.10 in the VM:

1. Launch VirtualBox - don't run your VM yet.
2. Select the VM you want to add your shared folder to.
1. Go to Settings > Shared Folder
2. You should only see "Machine Folder" in the list box if your VM is off. If you see "Transient Folder" in the list, that means your VM is running. Turn it off.
3. To the right of the list box, click on the Folder-with-Plus-Sign icon to add the folder.
4. Select the folder you want to use as your shared folder (create one if you like/necessary). For example, name the folder: "SharedFolder" (creative, I know). Check the Auto-mount option.
5. Accept, OK, and close the Settings window.
3. Run your VM.
4. Log in. (You can skip step 5 and go to step 6 - I'm just showing that the Shared Folder is not yet accessible at this point)
5. Open the File Browser/Explorer and try to access the shared folder at: /media/sfSharedFolder. You will get a "Permission denied" error. Don't panic - follow the next step.
6. Open a Terminal window.
7. Add yourself to the group: vboxsf.  E.g.
   ~$ usermod -g vboxsf
   You might need to become root before you can do it:
   ~$ sudo usermod -g vboxsf
8. Check that the group vboxsf has been added to your group list:
   ~$ groups
9. Now this is important: Log out, then log back in. You won't be able to access the shared folder if you don't.
10. After you log in, you should now be able to access your shared folder (/media/sfSharedFolder) without the "Permission Denied" error.

Hope that helps.