[AG-TECH] Problems uploading files

Vine, Derek A Derek.Vine at usd.edu
Tue Apr 27 10:23:58 CDT 2010


I am not sure if it is the same thing, but I too am having issues uploading files to our venue server, attached is the log that I sent to the list before but never heard anything back.  My setup is running on Windows server 2003.

Thanks,

Derek Vine

________________________________________
From: ag-tech-bounces at lists.mcs.anl.gov [ag-tech-bounces at lists.mcs.anl.gov] On Behalf Of Thomas Uram [turam at mcs.anl.gov]
Sent: Tuesday, April 27, 2010 10:06 AM
To: Christoph Willing
Cc: ag-tech at mcs.anl.gov
Subject: Re: [AG-TECH] Problems uploading files

The VenueClient.log shows no evidence of a problem. I suspect a server-side failure which prevents transmission of the ADD_DATA event, which would explain the client behavior.

Has anyone captured the problem in a VenueServer.log?

Tom

On Apr 27, 2010, at 6:42 AM, Christoph Willing wrote:

>
> On 27/04/2010, at 2:32 PM, John I. Quebedeaux, Jr wrote:
>
>> Hmmm... And ditto to Fedora 12, and 'ding' I may be having the same problem.
>> When I look for the files, they appear to be '0' sized though... But I
>> haven't tried a 'small' one yet.
>
>
> Thanks for adding to the evidence John.
>
> It seems there are two aspects to this problem. On the server side, the file is not received resulting in a zero size file. However I don't anymore think this is a data size issue - it just seemed that way to me at first. The AG's FTPSServer prepares to start but no data at all is actually exchanged between the client and the server.
>
> There is a client side problem too, that even when a file has uploaded correctly it may not appear in the venue client ui.
>
> Its a very weird problem. I spent most of today with an "old" system (in which server & client sides both work perfectly) and gradually built & installed updated packages so that they all match a current non-working system - figuring that the fault would manifest after one of these updates, identifying it as the culprit. "Unfortunately" the system still works at the end of all that - it seems none of the prime suspect packages is causing the problem. Tomorrow I'll either have to come with a new strategy or new suspect support packages. Does anyone want to suspect, say, openssl or even python itself?
>
>
> chris
>
>
>>> From: Christoph Willing <c.willing at uq.edu.au>
>>> Date: Tue, 27 Apr 2010 10:04:58 +1000
>>> To: Jesus Cea Oliva <jesus.cea at uca.es>
>>> Cc: <ag-tech at mcs.anl.gov>
>>> Subject: Re: [AG-TECH] Problems uploading files
>>>
>>> Jesus,
>>>
>>> Could you submit a bug report for this problem please? You can do that
>>> via the VenueClient Help menu.
>>>
>>>
>>> I have now been able to replicate the problem on several systems -
>>> Ubuntu, Slackware, Fedora & Debian. Interestingly older versions
>>> running on these systems seem to work OK i.e. when a venue server runs
>>> on the older systems, the problem is not evident. Unfortunately I
>>> don't have resources to test all Linux systems but so far, the last
>>> known versions to work correctly (a client can upload to a server
>>> running on the same system) are:
>>> Ubuntu Intrepid
>>> Slackware 12.2
>>> Fedora - unknown exactly but the problem is evident in Fedora 11
>>> Debian - Lenny partially works
>>>
>>> The Debian case is interesting because a Lenny client will not upload
>>> data to a Lenny server, yet a Jaunty client can upload to a Lenny
>>> server (even though it can't upload to a Jaunty server).
>>>
>>>
>>> At this stage, I can't determine whether the problem is due to AG
>>> toolkit code or to one of the supporting software packages; particular
>>> versions of zsi and m2crypto have caused most heartburn in the past -
>>> maybe they're implicated again. I think it will take some time to find
>>> and fix
>>>
>>>
>>> chris
>>>
>>>
>>> On 26/04/2010, at 12:51 AM, Jesus Cea Oliva wrote:
>>>
>>>> Hi Christoph, thanks for your indications :).
>>>>
>>>> I tried local VenueServers on machine, I mean, if I run Ubuntu, I
>>>> execute in the same machine a local VenueServer, then, I run
>>>> VenueClient and I try upload a file, the same with Windows 7.
>>>>
>>>> However, due to your mail, I've doing some tests:
>>>>
>>>> - On my Ubuntu system, I've entered at
>>>> https://vv3.ap-accessgrid.org:8000/Venues/default
>>>> , the Test Room, and I download files: VenueThermoClient.agpkg3 and
>>>> methanol.obj, and rename it adding X (metahnol.objX). Then I can
>>>> upload the files without problems to the server.
>>>>
>>>> I create a local network with 2 machines, with Windows 7 and Ubuntu.
>>>>
>>>> - Firstly I've executed a VenueServer on Ubuntu machine, then I've
>>>> run VenueClients in both machines. I can't upload files in the 2
>>>> machines (well you know, really, I can upload, but the Client
>>>> appears stop, reentering in the Venue I can see the files uploaded).
>>>> I've noticed other problem, when I try delete a file, VenueClient
>>>> don't refresh this command, the system don't delete the file.
>>>>
>>>>
>>>> - I've executed a VenueServer on Windows 7 machine, then I've run
>>>> VenueClients in both machines. Here, I can upload files without
>>>> problems, on Windows and Ubuntu systems.
>>>>
>>>> So, now, I know that I have problems with our VenueServer on Ubuntu
>>>> systems, I don't know if I have to configure something or if there
>>>> are problems about names, like ñ, spaces, etc (user name, path file,
>>>> etc, although my home path is /home/ercea and, there, is the
>>>> VenueServer's Data folder)
>>>>
>>>> Now I'll try change the system language to US and I'll repeat the
>>>> tests.
>>>>
>>>>
>>>> Thanks again and apologize for any inconveniences
>>>> Regards!
>>>>
>>>>
>>>>
>>>> El dia 25 abr 2010 12:11, Christoph Willing <c.willing at uq.edu.au>
>>>> escribió:
>>>> On 23/04/2010, at 6:24 PM, Jesus Cea Oliva wrote:
>>>>
>>>> Hi Crhistoph, thanks for your response :)
>>>>
>>>> I try upload files since 1 o 2 K (source codes, txts, etc.) and
>>>> files about 3-6 Mb (PDFs, ppts...) and, all cases I have the same
>>>> problem. The strange of this problem is that ocurrs randomly, and
>>>> sometimes, the same file that before appears not upload, now upload
>>>> (although, the majority cases, don't upload)
>>>>
>>>> Are you using the same venue server in each test case (Ubuntu &
>>>> Windows VenueClient)?
>>>>
>>>>
>>>> Are you using your own server or a "well known" server?
>>>> If these tests are with you own server, could you instead use the
>>>> APAG venue server at https://vv3.ap-accessgrid.org:8000/Venues/
>>>> default and then enter the Test venue. There are several files in
>>>> that venue; could you try to download the simple.py and methanol.obj
>>>> files from that venue and rename them e.g. simple.py -> simple.pyX
>>>> and methanol.obj -> methanol.objX. Now upload te renamed files back
>>>> into the Test venue. Do they both upload as expected?
>>>>
>>>>
>>>> FYI, although I thought I had replicated the problem, I now find
>>>> that it actually works correctly. My first hurried tests only
>>>> appeared to replicate your symptoms - because I didn't wait long
>>>> enogh for the upload to complete. However I have now performed the
>>>> tests again and all files upload correctly - and the venue client
>>>> interface shows the new data correctly in all cases.
>>>>
>>>> I believe the previous test appeared not to update the venue client
>>>> interface because I was performing the tests from home using an ADSL
>>>> connection - therefore the initial download was quite fast. The
>>>> upload was slower because of the much lower upstream bandwidth of my
>>>> ADSL connection. My guess is that the first portion of upload
>>>> _appears_ fast because the venue client is just filling up the local
>>>> TCP system buffer. However that buffer then has to be transferred
>>>> (via slow ADSL uplink) to the venue server - which manifests as a
>>>> fast intitial movement of the progress bar that then appears to
>>>> freeze. The "freeze" is really just a pause while contents of the
>>>> TCP buffer is being transferred across the network.
>>>>
>>>> In my case the simple.py file, being small, transfers very quickly.
>>>> The methanol.obj file is about 11M - I guess this is just a bit
>>>> larger than my system network buffer size - and takes a lot longer
>>>> to transfer.
>>>>
>>>> Based on redoing the tests (using the APAG veneu server), I cannot
>>>> replicate the problem.
>>>>
>>>> Could you confirm whether you still have the problem when you use
>>>> the APAG venue server?
>>>>
>>>>
>>>> chris
>>>>
>>>>
>>>> On Windows I try similar files (somes files are the same), only
>>>> don't work once due to a very big file. As I saw that I can upload
>>>> files without problems... I try all file types. So I upload a file
>>>> about 100Mb and don't work xD.
>>>>
>>>> Thanks for your help :). Regards!
>>>>
>>>> El dia 23 abr 2010 02:35, Christoph Willing <c.willing at uq.edu.au>
>>>> escribió:
>>>> I wonder if this is related to the size of the data object. I just
>>>> tried (with Ubuntu 9.10 64bit) and found that a small file (6K)
>>>> there was no problem. With a larger file (12M) I saw exactly the
>>>> same problem - upload appears not to complete but has actually
>>>> succeeded. Re-entering the venue shows that the new data is there.
>>>>
>>>> In your Windows test, are you uploading the same file (or something
>>>> of similar size) as in the Ubuntu case?
>>>>
>>>> I will now be away for a few days so I can't do any more
>>>> investigation myself until I return. Maybe others can look into this
>>>> in the meantime ...
>>>>
>>>>
>>>> chris
>>>>
>>>>
>>>> On 23/04/2010, at 5:28 AM, Jesus Cea Oliva wrote:
>>>>
>>>> Thanks Tom for your fast response :).
>>>>
>>>> Well, atach the VenueClient log in this email, I can't see any
>>>> problems :S.
>>>>
>>>>
>>>>
>>>> 04/22/10 21:16:50 140471913264880 VenueClient VenueClientUI.py: 1308
>>>> DEBUG VenueClientUI.AddDataCB: Trying to upload to
>>>> 'ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007'
>>>> 04/22/10 21:17:00 140471913264880 VenueClient VenueClientUI.py: 1395
>>>> DEBUG AddDataCB: URI of parent is
>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>> VenueClientController.py:821 DEBUG In
>>>> VenueClientController.UploadVenueFiles
>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>> VenueClientController.py:822 DEBUG fileList = [u'/home/ercea/
>>>> Descargas/DSD-Tutorial-3.pdf']
>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>> VenueClientController.py:848 DEBUG Serverpath:
>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>> VenueClientController.py:857 DEBUG Have args, creating thread, url:
>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>> , files: [u'/home/ercea/Descargas/DSD-Tutorial-3.pdf']
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:944 DEBUG Upload: getting identity
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:949 DEBUG Got identity <?xml
>>>> version="1.0" ? ><Subject><Subject auth_data="" auth_type="x509"
>>>> name="/O=Access Grid/O=Argonne National Laboratory/OU=Futures Lab
>>>> Anonymous Authority/CN=Anonymous User
>>>> 17130398eef9fd82cba8dc9d845a1768"/></ Subject>
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:950 DEBUG get_ident_and_upload: Upload URL
>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>> ftps:
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>> localhost:8006
>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>> 7f00010133381d7afaba1c148fcabe1007
>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1267 INFO
>>>> UploadFiles: ftps://localhost:
>>>> 8006/7f00010133381d7afaba1c148fcabe1007 [u'/home/ercea/Descargas/
>>>> DSD- Tutorial-3.pdf']
>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1271 DEBUG
>>>> UploadFiles: ftps://localhost:
>>>> 8006/7f00010133381d7afaba1c148fcabe1007 /home/ercea/Descargas/DSD-
>>>> Tutorial-3.pdf [u'/home/ercea/Descargas/ DSD-Tutorial-3.pdf']
>>>> 04/22/10 21:17:00 140471479552272 FTPSClient FTPSClient.py:141 DEBUG
>>>> Entered FTPSUploadFile: localfile=/home/ercea/Descargas/DSD-
>>>> Tutorial-3.pdf url=ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>> VenueClientController.py:862 DEBUG Started thread
>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:605
>>>> DEBUG Calling Heartbeat, time now: 1271963840
>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:629
>>>> DEBUG Next Heartbeat needed within 36s
>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:639
>>>> DEBUG heartBeatCounter = 2
>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:605
>>>> DEBUG Calling Heartbeat, time now: 1271963876
>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:629
>>>> DEBUG Next Heartbeat needed within 36s
>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:639
>>>> DEBUG heartBeatCounter = 3
>>>>
>>>>
>>>> Here increment heartBeatCounter +1.
>>>>
>>>> Ah, I have this problem, in the same network, on ubuntu system
>>>> (Ubuntu 9.10 64 bits), but I've tested on Windows 7 and I can upload
>>>> files without problems.
>>>>
>>>> Regards!
>>>>
>>>>
>>>> El dia 22 abr 2010 18:29, Thomas Uram <turam at mcs.anl.gov> escribió:
>>>> Hi Jesus:
>>>>
>>>> There is probably some indication of the problem in the
>>>> VenueClient.log file. If you submit a bug report from the Help menu
>>>> after this happens, a recent portion of the VenueClient.log file
>>>> will be included with the report, and we can review it. If you want
>>>> to look yourself, the log file resides in the following locations
>>>> based on platform:
>>>> MacOS, Linux: ~/.AccessGrid3/Logs
>>>> Windows: c:\documents and settings\username\application data
>>>> \AccessGrid3\Logs (adjusted for your locale, of course)
>>>> Tom
>>>>
>>>> On Apr 22, 2010, at 11:11 AM, Jesus Cea Oliva wrote:
>>>>
>>>> Hi list again, sorry for my constants problems xD.
>>>>
>>>> I'm trying upload files to our VenueServer and my problem is
>>>> strange. Firstly, I've created a VenueServer locally, for testing.
>>>>
>>>> Well, when I try to upload a file, I can see the message on the
>>>> VenueClient "Uploading file X" and a progressbar, but, when the
>>>> progressbar is near to the end, stops and list of files (Data
>>>> section) don't refresh. So I beleived that the file can't upload.
>>>>
>>>> But, really, the file has uploaded, because I can see the folder
>>>> Data of my local VenueServer, and the file is copied in this folder.
>>>> In this case, if I reset VenueServer and VenueClient, then, I can
>>>> see the files in Data section.
>>>>
>>>> So I think that the file uploads fine but, the list of files don't
>>>> refresh and the file uploaded don't appears.
>>>>
>>>> I must say that, sometimes, I can upload a file so I don't know
>>>> whats really the problem (ports closed? firewall? path or name
>>>> files? file size?) .
>>>>
>>>> Thanks again and apologize for any inconveniences
>>>>
>>>> Regards!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> <VenueClient.log>
>>>>
>>>> Christoph Willing +61 7 3365 8316
>>>> QCIF Access Grid Manager
>>>> University of Queensland
>>>>
>>>> Christoph Willing +61 7 3365 8316
>>>> QCIF Access Grid Manager
>>>> University of Queensland
>>>>
>>>>
>>>
>>> Christoph Willing                       +61 7 3365 8316
>>> QCIF Access Grid Manager
>>> University of Queensland
>>>
>>
>
> Christoph Willing                       +61 7 3365 8316
> QCIF Access Grid Manager
> University of Queensland
>


More information about the ag-tech mailing list