[Swift-devel] Re: Getting VMs from FG for use with swift

Tue Jun 7 14:39:20 CDT 2011

Hello John,

I have attached a quickstart guide I wrote on how to get Swift working with
futuregrid by using the new Swift coaster service scripts. This will require
the latest development version of Swift. Instructions on how to
download/install are in the document.

Please let me know if you have any questions, if anything is unclear, or if
you run into any problems. Thank you!

Regards,
David

On Tue, May 24, 2011 at 3:18 PM, John Bresnahan <bresnaha at mcs.anl.gov>wrote:

> The GPFS server on the FG cluster hotel died yesterday so I cannot get you
> your credentials.  I'll get back to you when it is up again.  Once it is
> back the process for getting the needed access keys is described here:
>
> https://portal.futuregrid.org/tutorials/nimbus
>
>
> On 05/23/2011 05:24 AM, David Kelly wrote:
>
>> Hi John,
>>
>> I now have a futuregrid account and am added to a project. I am now trying
>> to get our scripts
>> working together.
>>
>> I ran into a few problems at first when trying to run the futuregrid
>> scripts. On the first system I
>> tried I was getting a traceback. It is possible that the system I was
>> using has older versions of
>> some of the needed libraries. Then I tried it on a more system that is
>> more frequently updated - my
>> laptop running Ubuntu 10.10.  It needed a newer version of the Python
>> crypto tools installed, so I
>> installed that (and the python development libraries) and that part seems
>> fine now.
>>
>> I am now up to the point of the install script where it is trying to
>> register keys, but it is
>> failing. My guess is that I need to change FUTUREGRID_IAAS_ACCESS_KEY and
>> FUTUREGRID_IAAS_SECRET_KEY
>> in env.sh. I'm not sure what these should be exactly. Are these the
>> contents of my ssh keys, an ssh
>> key and a passphrase, or some other type of security? I've tried a few
>> combinations of different
>> things but haven't had much luck yet.
>>
>> Thanks!
>>
>> Regards,
>> David
>>
>>
>> Traceback from earlier:
>> Installing setuptools.......................done.
>>   Complete output from command /autonfs/home/davidk/swift-vm-...ython
>> /autonfs/home/davidk/swift-vm-...stall pip:
>>   Searching for pip
>> Reading http://pypi.python.org/simple/pip/
>> Reading http://pip.openplans.org
>> Reading http://www.pip-installer.org
>> Best match: pip 1.0.1
>> Downloading
>>
>> http://pypi.python.org/packages/source/p/pip/pip-1.0.1.tar.gz#md5=28dcc70225e5bf925532abc5b087a94b
>> Processing pip-1.0.1.tar.gz
>> Running pip-1.0.1/setup.py -q bdist_egg --dist-dir
>> /tmp/easy_install-GHsjHX/pip-1.0.1/egg-dist-tmp-rXjQ7L
>> Traceback (most recent call last):
>>   File "/autonfs/home/davidk/swift-vm-boot/ve/bin/easy_install", line 8,
>> in <module>
>>     load_entry_point('setuptools==0.6c11', 'console_scripts',
>> 'easy_install')()
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 1712, in main
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 1700, in with_ei_usage
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 1716, in <lambda>
>>   File "/soft/python-2.6.1-r1/lib/python2.6/distutils/core.py", line 152,
>> in setup
>>     dist.run_commands()
>>   File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 975,
>> in run_commands
>>     self.run_command(cmd)
>>   File "/soft/python-2.6.1-r1/lib/python2.6/distutils/dist.py", line 995,
>> in run_command
>>     cmd_obj.run()
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 211, in run
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 446, in easy_install
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 476, in install_item
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 655, in install_eggs
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 930, in build_and_install
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/command/easy_install.py",
>> line 919, in run_setup
>>   File
>>
>> "/autonfs/home/davidk/swift-vm-boot/ve/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/setuptools/sandbox.py",
>> line 52, in run_setup
>> AttributeError: 'module' object has no attribute '__getstate__'
>> ----------------------------------------
>> Traceback (most recent call last):
>>   File "bin/virtualenv.py", line 1647, in <module>
>>     main()
>>   File "bin/virtualenv.py", line 558, in main
>>     prompt=options.prompt)
>>   File "bin/virtualenv.py", line 656, in create_environment
>>     install_pip(py_executable)
>>   File "bin/virtualenv.py", line 415, in install_pip
>>     filter_stdout=_filter_setup)
>>   File "bin/virtualenv.py", line 624, in call_subprocess
>>     % (cmd_desc, proc.returncode))
>> OSError: Command /autonfs/home/davidk/swift-vm-...ython
>> /autonfs/home/davidk/swift-vm-...stall pip
>> failed with error code 1
>> Failed to created the needed python virtual environment
>>
>> On Fri, May 20, 2011 at 7:20 PM, John Bresnahan <bresnaha at mcs.anl.gov<mailto:
>> bresnaha at mcs.anl.gov>>
>>
>> wrote:
>>
>>    Our phone call today left me motiviated to show you guys how easy it is
>> to get virtual machines
>>    for use with swift on FutureGrid.
>>
>>    I made some small scripts around the Nimbus tool cloudinitd.  The
>> scripts just make installing
>>    the software and running it trivial.  With a single command you can get
>> N VMs from the
>>    FutureGrid Nimbus clouds (N can be on the order of hundreds).  When the
>> tool is done it outputs
>>    a line separated list of hostnames.  All of these hostnames have root
>> access available via your
>>    ~/.ssh/id_rsa keys.
>>
>>    If/when you have FutureGrid credentials, untar the attachment and give
>> it a try.  There are a
>>    few minor configurations needed:
>>
>>
>>    1) edit the file env.sh and set your FutureGrid security credentials:
>>
>>    % cat env.sh
>>    export FUTUREGRID_IAAS_ACCESS_KEY=XXXXXXXXXXXXXXXXXX
>>    export FUTUREGRID_IAAS_SECRET_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>>
>>    export FUTUREGRID_HOTEL_NODES=2
>>    export FUTUREGRID_SIERRA_NODES=2
>>
>>    You can also change the value '2' to be whatever number of VMs you
>> want.
>>
>>
>>    2) install it on your system.  (this single command downloads and
>> installs everything you need
>>    under the cwd):
>>
>>    % ./install.sh
>>
>>    3) boot the VMs
>>    % ./bin/bootit.sh.
>> You will see much status output, but the last several lines will be the
>> hostnames acquired from
>>    the cloud.
>>
>>    Let me know when you guys are ready to check this out!
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110607/13e453a5/attachment.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/swift-devel/attachments/20110607/13e453a5/attachment-0001.html>