[Swift-devel] Re: Getting VMs from FG for use with swift

Thu Jun 2 13:42:19 CDT 2011

I have not forgotten about you guys, but the gpfs file system on hotel.futuregrid.org continues to 
have problems.  I arrive in Chicago on Friday so we should have something straightened out next week.

Sorry about the delay.

John

On 05/23/2011 05:24 AM, David Kelly wrote:
> Hi John,
>
> I now have a futuregrid account and am added to a project. I am now trying to get our scripts
> working together.
>
> I ran into a few problems at first when trying to run the futuregrid scripts. On the first system I
> tried I was getting a traceback. It is possible that the system I was using has older versions of
> some of the needed libraries. Then I tried it on a more system that is more frequently updated - my
> laptop running Ubuntu 10.10.  It needed a newer version of the Python crypto tools installed, so I
> installed that (and the python development libraries) and that part seems fine now.
>
> I am now up to the point of the install script where it is trying to register keys, but it is
> failing. My guess is that I need to change FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY
> in env.sh. I'm not sure what these should be exactly. Are these the contents of my ssh keys, an ssh
> key and a passphrase, or some other type of security? I've tried a few combinations of different
> things but haven't had much luck yet.
>
> Thanks!
>
> Regards,
> David
>
>
> Traceback from earlier:
> Installing setuptools.......................done.
>    Complete output from command /autonfs/home/davidk/swift-vm-...ython
> /autonfs/home/davidk/swift-vm-...stall pip:
>    Searching for pip
> Reading http://pypi.python.org/simple/pip/
> Reading http://pip.openplans.org
> Reading http://www.pip-installer.org
> Best match: pip 1.0.1
> Downloading
> http://pypi.python.org/packages/source/p/pip/pip-1.0.1.tar.gz#md5=28dcc70225e5bf925532abc5b087a94b
> Processing pip-1.0.1.tar.gz
> Running pip-1.0.1/setup.py -q bdist_egg --dist-dir
> /tmp/easy_install-GHsjHX/pip-1.0.1/egg-dist-tmp-rXjQ7L
> Traceback (most recent call last):
>    File "/autonfs/home/davidk/swift-vm-boot/ve/bin/easy_install", line 8, in <module>
>      load_entry_point('setuptools==0.6c11', 'console_scripts', 'easy_install')()
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 1712, in main
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 1700, in with_ei_usage
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 1716, in <lambda>
>    File "/soft/python-2.6.1-r1/lib/python2.6/distutils/core.py", line 152, in setup
>      dist.run_commands()
>    File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 975, in run_commands
>      self.run_command(cmd)
>    File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 995, in run_command
>      cmd_obj.run()
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 211, in run
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 446, in easy_install
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 476, in install_item
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 655, in install_eggs
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 930, in build_and_install
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
> line 919, in run_setup
>    File
> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/sandbox.py",
> line 52, in run_setup
> AttributeError: 'module' object has no attribute '__getstate__'
> ----------------------------------------
> Traceback (most recent call last):
>    File "bin/virtualenv.py", line 1647, in <module>
>      main()
>    File "bin/virtualenv.py", line 558, in main
>      prompt=options.prompt)
>    File "bin/virtualenv.py", line 656, in create_environment
>      install_pip(py_executable)
>    File "bin/virtualenv.py", line 415, in install_pip
>      filter_stdout=_filter_setup)
>    File "bin/virtualenv.py", line 624, in call_subprocess
>      % (cmd_desc, proc.returncode))
> OSError: Command /autonfs/home/davidk/swift-vm-...ython /autonfs/home/davidk/swift-vm-...stall pip
> failed with error code 1
> Failed to created the needed python virtual environment
>
> On Fri, May 20, 2011 at 7:20 PM, John Bresnahan <bresnaha at mcs.anl.gov <mailto:bresnaha at mcs.anl.gov>>
> wrote:
>
>     Our phone call today left me motiviated to show you guys how easy it is to get virtual machines
>     for use with swift on FutureGrid.
>
>     I made some small scripts around the Nimbus tool cloudinitd.  The scripts just make installing
>     the software and running it trivial.  With a single command you can get N VMs from the
>     FutureGrid Nimbus clouds (N can be on the order of hundreds).  When the tool is done it outputs
>     a line separated list of hostnames.  All of these hostnames have root access available via your
>     ~/.ssh/id_rsa keys.
>
>     If/when you have FutureGrid credentials, untar the attachment and give it a try.  There are a
>     few minor configurations needed:
>
>
>     1) edit the file env.sh and set your FutureGrid security credentials:
>
>     % cat env.sh
>     export FUTUREGRID_IAAS_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
>     export FUTUREGRID_IAAS_SECRET_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>     export FUTUREGRID_HOTEL_NODES=2
>     export FUTUREGRID_SIERRA_NODES=2
>
>     You can also change the value '2' to be whatever number of VMs you want.
>
>
>     2) install it on your system.  (this single command downloads and installs everything you need
>     under the cwd):
>
>     % ./install.sh
>
>     3) boot the VMs
>     % ./bin/bootit.sh.
>
>     You will see much status output, but the last several lines will be the hostnames acquired from
>     the cloud.
>
>     Let me know when you guys are ready to check this out!
>
>