[Swift-devel] Re: Getting VMs from FG for use with swift

[David Kelly]

Hi John,

I now have a futuregrid account and am added to a project. I am now trying
to get our scripts working together.

I ran into a few problems at first when trying to run the futuregrid
scripts. On the first system I tried I was getting a traceback. It is
possible that the system I was using has older versions of some of the
needed libraries. Then I tried it on a more system that is more frequently
updated - my laptop running Ubuntu 10.10.  It needed a newer version of the
Python crypto tools installed, so I installed that (and the python
development libraries) and that part seems fine now.

I am now up to the point of the install script where it is trying to
register keys, but it is failing. My guess is that I need to change
FUTUREGRID_IAAS_ACCESS_KEY and FUTUREGRID_IAAS_SECRET_KEY in env.sh. I'm not
sure what these should be exactly. Are these the contents of my ssh keys, an
ssh key and a passphrase, or some other type of security? I've tried a few
combinations of different things but haven't had much luck yet.

Thanks!

Regards,
David


Traceback from earlier:
Installing setuptools.......................done.
  Complete output from command /autonfs/home/davidk/swift-vm-...ython
/autonfs/home/davidk/swift-vm-...stall pip:
  Searching for pip
Reading http://pypi.python.org/simple/pip/
Reading http://pip.openplans.org
Reading http://www.pip-installer.org
Best match: pip 1.0.1
Downloading
http://pypi.python.org/packages/source/p/pip/pip-1.0.1.tar.gz#md5=28dcc70225e5bf925532abc5b087a94b
Processing pip-1.0.1.tar.gz
Running pip-1.0.1/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-GHsjHX/pip-1.0.1/egg-dist-tmp-rXjQ7L
Traceback (most recent call last):
  File "/autonfs/home/davidk/swift-vm-boot/ve/bin/easy_install", line 8, in
<module>
    load_entry_point('setuptools==0.6c11', 'console_scripts',
'easy_install')()
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 1712, in main
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 1700, in with_ei_usage
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 1716, in <lambda>
  File "/soft/python-2.6.1-r1/lib/python2.6/distutils/core.py", line 152, in
setup
    dist.run_commands()
  File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 975, in
run_commands
    self.run_command(cmd)
  File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 995, in
run_command
    cmd_obj.run()
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 211, in run
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 446, in easy_install
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 476, in install_item
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 655, in install_eggs
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 930, in build_and_install
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
line 919, in run_setup
  File
"/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/sandbox.py",
line 52, in run_setup
AttributeError: 'module' object has no attribute '__getstate__'
----------------------------------------
Traceback (most recent call last):
  File "bin/virtualenv.py", line 1647, in <module>
    main()
  File "bin/virtualenv.py", line 558, in main
    prompt=options.prompt)
  File "bin/virtualenv.py", line 656, in create_environment
    install_pip(py_executable)
  File "bin/virtualenv.py", line 415, in install_pip
    filter_stdout=_filter_setup)
  File "bin/virtualenv.py", line 624, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /autonfs/home/davidk/swift-vm-...ython
/autonfs/home/davidk/swift-vm-...stall pip failed with error code 1
Failed to created the needed python virtual environment

On Fri, May 20, 2011 at 7:20 PM, John Bresnahan <bresnaha at mcs.anl.gov>wrote:

> Our phone call today left me motiviated to show you guys how easy it is to
> get virtual machines for use with swift on FutureGrid.
>
> I made some small scripts around the Nimbus tool cloudinitd.  The scripts
> just make installing the software and running it trivial.  With a single
> command you can get N VMs from the FutureGrid Nimbus clouds (N can be on the
> order of hundreds).  When the tool is done it outputs a line separated list
> of hostnames.  All of these hostnames have root access available via your
> ~/.ssh/id_rsa keys.
>
> If/when you have FutureGrid credentials, untar the attachment and give it a
> try.  There are a few minor configurations needed:
>
>
> 1) edit the file env.sh and set your FutureGrid security credentials:
>
> % cat env.sh
> export FUTUREGRID_IAAS_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
> export FUTUREGRID_IAAS_SECRET_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> export FUTUREGRID_HOTEL_NODES=2
> export FUTUREGRID_SIERRA_NODES=2
>
> You can also change the value '2' to be whatever number of VMs you want.
>
>
> 2) install it on your system.  (this single command downloads and installs
> everything you need under the cwd):
>
> % ./install.sh
>
> 3) boot the VMs
> % ./bin/bootit.sh.
>
> You will see much status output, but the last several lines will be the
> hostnames acquired from the cloud.
>
> Let me know when you guys are ready to check this out!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110523/6059b1bc/attachment.html>