[AG-TECH] Problems uploading files
Vine, Derek A
Derek.Vine at usd.edu
Tue Apr 27 15:38:46 CDT 2010
At least in my case, I connect to the venue locally from the venue server, I can upload files like expected. Connecting from any other node will not allow upload of files, at least not "live". Sometimes after you try to upload a file and leave and come back the file is there.
Any help is appreciated.
--Derek
-----Original Message-----
From: ag-tech-bounces at lists.mcs.anl.gov [mailto:ag-tech-bounces at lists.mcs.anl.gov] On Behalf Of Christoph Willing
Sent: Tuesday, April 27, 2010 3:27 PM
To: Vine, Derek A
Cc: ag-tech at mcs.anl.gov
Subject: Re: [AG-TECH] Problems uploading files
On 28/04/2010, at 1:31 AM, Vine, Derek A wrote:
> would help if I attached the file...
>
> ________________________________________
> From: ag-tech-bounces at lists.mcs.anl.gov [ag-tech-bounces at lists.mcs.anl.gov
> ] On Behalf Of Vine, Derek A [Derek.Vine at usd.edu]
> Sent: Tuesday, April 27, 2010 10:23 AM
> To: Thomas Uram; Christoph Willing
> Cc: ag-tech at mcs.anl.gov
> Subject: Re: [AG-TECH] Problems uploading files
>
> I am not sure if it is the same thing, but I too am having issues
> uploading files to our venue server, attached is the log that I sent
> to the list before but never heard anything back. My setup is
> running on Windows server 2003.
Derek,
I wondered whether they were connected as I was looking around for
similar cases but assumed they were different since you mentioned
Windows - the OP in this thread said that Windows was OK for them.
However looking at your log file, I see that its almost identical to
the excerpt I just sent to the list a few minutes ago. The one
difference that I see is that you have an additional line:
02/03/10 09:33:04 4180 FTPSServer ftps_server.py:275 INFO
recv: closing channel <AccessGrid.FTPS.ftps_server.ftp_tls_channel
connected 206.176.1.99:1119 at 0x2cc23c8> (0, 'Error')
I don't have any recognition that there's an error after "INFO ==>
227 Entering Passive Mode".
chris
> ________________________________________
> From: ag-tech-bounces at lists.mcs.anl.gov [ag-tech-bounces at lists.mcs.anl.gov
> ] On Behalf Of Thomas Uram [turam at mcs.anl.gov]
> Sent: Tuesday, April 27, 2010 10:06 AM
> To: Christoph Willing
> Cc: ag-tech at mcs.anl.gov
> Subject: Re: [AG-TECH] Problems uploading files
>
> The VenueClient.log shows no evidence of a problem. I suspect a
> server-side failure which prevents transmission of the ADD_DATA
> event, which would explain the client behavior.
>
> Has anyone captured the problem in a VenueServer.log?
>
> Tom
>
> On Apr 27, 2010, at 6:42 AM, Christoph Willing wrote:
>
>>
>> On 27/04/2010, at 2:32 PM, John I. Quebedeaux, Jr wrote:
>>
>>> Hmmm... And ditto to Fedora 12, and 'ding' I may be having the
>>> same problem.
>>> When I look for the files, they appear to be '0' sized though...
>>> But I
>>> haven't tried a 'small' one yet.
>>
>>
>> Thanks for adding to the evidence John.
>>
>> It seems there are two aspects to this problem. On the server side,
>> the file is not received resulting in a zero size file. However I
>> don't anymore think this is a data size issue - it just seemed that
>> way to me at first. The AG's FTPSServer prepares to start but no
>> data at all is actually exchanged between the client and the server.
>>
>> There is a client side problem too, that even when a file has
>> uploaded correctly it may not appear in the venue client ui.
>>
>> Its a very weird problem. I spent most of today with an "old"
>> system (in which server & client sides both work perfectly) and
>> gradually built & installed updated packages so that they all match
>> a current non-working system - figuring that the fault would
>> manifest after one of these updates, identifying it as the culprit.
>> "Unfortunately" the system still works at the end of all that - it
>> seems none of the prime suspect packages is causing the problem.
>> Tomorrow I'll either have to come with a new strategy or new
>> suspect support packages. Does anyone want to suspect, say, openssl
>> or even python itself?
>>
>>
>> chris
>>
>>
>>>> From: Christoph Willing <c.willing at uq.edu.au>
>>>> Date: Tue, 27 Apr 2010 10:04:58 +1000
>>>> To: Jesus Cea Oliva <jesus.cea at uca.es>
>>>> Cc: <ag-tech at mcs.anl.gov>
>>>> Subject: Re: [AG-TECH] Problems uploading files
>>>>
>>>> Jesus,
>>>>
>>>> Could you submit a bug report for this problem please? You can do
>>>> that
>>>> via the VenueClient Help menu.
>>>>
>>>>
>>>> I have now been able to replicate the problem on several systems -
>>>> Ubuntu, Slackware, Fedora & Debian. Interestingly older versions
>>>> running on these systems seem to work OK i.e. when a venue server
>>>> runs
>>>> on the older systems, the problem is not evident. Unfortunately I
>>>> don't have resources to test all Linux systems but so far, the last
>>>> known versions to work correctly (a client can upload to a server
>>>> running on the same system) are:
>>>> Ubuntu Intrepid
>>>> Slackware 12.2
>>>> Fedora - unknown exactly but the problem is evident in Fedora 11
>>>> Debian - Lenny partially works
>>>>
>>>> The Debian case is interesting because a Lenny client will not
>>>> upload
>>>> data to a Lenny server, yet a Jaunty client can upload to a Lenny
>>>> server (even though it can't upload to a Jaunty server).
>>>>
>>>>
>>>> At this stage, I can't determine whether the problem is due to AG
>>>> toolkit code or to one of the supporting software packages;
>>>> particular
>>>> versions of zsi and m2crypto have caused most heartburn in the
>>>> past -
>>>> maybe they're implicated again. I think it will take some time to
>>>> find
>>>> and fix
>>>>
>>>>
>>>> chris
>>>>
>>>>
>>>> On 26/04/2010, at 12:51 AM, Jesus Cea Oliva wrote:
>>>>
>>>>> Hi Christoph, thanks for your indications :).
>>>>>
>>>>> I tried local VenueServers on machine, I mean, if I run Ubuntu, I
>>>>> execute in the same machine a local VenueServer, then, I run
>>>>> VenueClient and I try upload a file, the same with Windows 7.
>>>>>
>>>>> However, due to your mail, I've doing some tests:
>>>>>
>>>>> - On my Ubuntu system, I've entered at
>>>>> https://vv3.ap-accessgrid.org:8000/Venues/default
>>>>> , the Test Room, and I download files: VenueThermoClient.agpkg3
>>>>> and
>>>>> methanol.obj, and rename it adding X (metahnol.objX). Then I can
>>>>> upload the files without problems to the server.
>>>>>
>>>>> I create a local network with 2 machines, with Windows 7 and
>>>>> Ubuntu.
>>>>>
>>>>> - Firstly I've executed a VenueServer on Ubuntu machine, then I've
>>>>> run VenueClients in both machines. I can't upload files in the 2
>>>>> machines (well you know, really, I can upload, but the Client
>>>>> appears stop, reentering in the Venue I can see the files
>>>>> uploaded).
>>>>> I've noticed other problem, when I try delete a file, VenueClient
>>>>> don't refresh this command, the system don't delete the file.
>>>>>
>>>>>
>>>>> - I've executed a VenueServer on Windows 7 machine, then I've run
>>>>> VenueClients in both machines. Here, I can upload files without
>>>>> problems, on Windows and Ubuntu systems.
>>>>>
>>>>> So, now, I know that I have problems with our VenueServer on
>>>>> Ubuntu
>>>>> systems, I don't know if I have to configure something or if there
>>>>> are problems about names, like ñ, spaces, etc (user name, path
>>>>> file,
>>>>> etc, although my home path is /home/ercea and, there, is the
>>>>> VenueServer's Data folder)
>>>>>
>>>>> Now I'll try change the system language to US and I'll repeat the
>>>>> tests.
>>>>>
>>>>>
>>>>> Thanks again and apologize for any inconveniences
>>>>> Regards!
>>>>>
>>>>>
>>>>>
>>>>> El dia 25 abr 2010 12:11, Christoph Willing <c.willing at uq.edu.au>
>>>>> escribió:
>>>>> On 23/04/2010, at 6:24 PM, Jesus Cea Oliva wrote:
>>>>>
>>>>> Hi Crhistoph, thanks for your response :)
>>>>>
>>>>> I try upload files since 1 o 2 K (source codes, txts, etc.) and
>>>>> files about 3-6 Mb (PDFs, ppts...) and, all cases I have the same
>>>>> problem. The strange of this problem is that ocurrs randomly, and
>>>>> sometimes, the same file that before appears not upload, now
>>>>> upload
>>>>> (although, the majority cases, don't upload)
>>>>>
>>>>> Are you using the same venue server in each test case (Ubuntu &
>>>>> Windows VenueClient)?
>>>>>
>>>>>
>>>>> Are you using your own server or a "well known" server?
>>>>> If these tests are with you own server, could you instead use the
>>>>> APAG venue server at https://vv3.ap-accessgrid.org:8000/Venues/
>>>>> default and then enter the Test venue. There are several files in
>>>>> that venue; could you try to download the simple.py and
>>>>> methanol.obj
>>>>> files from that venue and rename them e.g. simple.py -> simple.pyX
>>>>> and methanol.obj -> methanol.objX. Now upload te renamed files
>>>>> back
>>>>> into the Test venue. Do they both upload as expected?
>>>>>
>>>>>
>>>>> FYI, although I thought I had replicated the problem, I now find
>>>>> that it actually works correctly. My first hurried tests only
>>>>> appeared to replicate your symptoms - because I didn't wait long
>>>>> enogh for the upload to complete. However I have now performed the
>>>>> tests again and all files upload correctly - and the venue client
>>>>> interface shows the new data correctly in all cases.
>>>>>
>>>>> I believe the previous test appeared not to update the venue
>>>>> client
>>>>> interface because I was performing the tests from home using an
>>>>> ADSL
>>>>> connection - therefore the initial download was quite fast. The
>>>>> upload was slower because of the much lower upstream bandwidth
>>>>> of my
>>>>> ADSL connection. My guess is that the first portion of upload
>>>>> _appears_ fast because the venue client is just filling up the
>>>>> local
>>>>> TCP system buffer. However that buffer then has to be transferred
>>>>> (via slow ADSL uplink) to the venue server - which manifests as a
>>>>> fast intitial movement of the progress bar that then appears to
>>>>> freeze. The "freeze" is really just a pause while contents of the
>>>>> TCP buffer is being transferred across the network.
>>>>>
>>>>> In my case the simple.py file, being small, transfers very
>>>>> quickly.
>>>>> The methanol.obj file is about 11M - I guess this is just a bit
>>>>> larger than my system network buffer size - and takes a lot longer
>>>>> to transfer.
>>>>>
>>>>> Based on redoing the tests (using the APAG veneu server), I cannot
>>>>> replicate the problem.
>>>>>
>>>>> Could you confirm whether you still have the problem when you use
>>>>> the APAG venue server?
>>>>>
>>>>>
>>>>> chris
>>>>>
>>>>>
>>>>> On Windows I try similar files (somes files are the same), only
>>>>> don't work once due to a very big file. As I saw that I can upload
>>>>> files without problems... I try all file types. So I upload a file
>>>>> about 100Mb and don't work xD.
>>>>>
>>>>> Thanks for your help :). Regards!
>>>>>
>>>>> El dia 23 abr 2010 02:35, Christoph Willing <c.willing at uq.edu.au>
>>>>> escribió:
>>>>> I wonder if this is related to the size of the data object. I just
>>>>> tried (with Ubuntu 9.10 64bit) and found that a small file (6K)
>>>>> there was no problem. With a larger file (12M) I saw exactly the
>>>>> same problem - upload appears not to complete but has actually
>>>>> succeeded. Re-entering the venue shows that the new data is there.
>>>>>
>>>>> In your Windows test, are you uploading the same file (or
>>>>> something
>>>>> of similar size) as in the Ubuntu case?
>>>>>
>>>>> I will now be away for a few days so I can't do any more
>>>>> investigation myself until I return. Maybe others can look into
>>>>> this
>>>>> in the meantime ...
>>>>>
>>>>>
>>>>> chris
>>>>>
>>>>>
>>>>> On 23/04/2010, at 5:28 AM, Jesus Cea Oliva wrote:
>>>>>
>>>>> Thanks Tom for your fast response :).
>>>>>
>>>>> Well, atach the VenueClient log in this email, I can't see any
>>>>> problems :S.
>>>>>
>>>>>
>>>>>
>>>>> 04/22/10 21:16:50 140471913264880 VenueClient VenueClientUI.py:
>>>>> 1308
>>>>> DEBUG VenueClientUI.AddDataCB: Trying to upload to
>>>>> 'ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007'
>>>>> 04/22/10 21:17:00 140471913264880 VenueClient VenueClientUI.py:
>>>>> 1395
>>>>> DEBUG AddDataCB: URI of parent is
>>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>>> VenueClientController.py:821 DEBUG In
>>>>> VenueClientController.UploadVenueFiles
>>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>>> VenueClientController.py:822 DEBUG fileList = [u'/home/ercea/
>>>>> Descargas/DSD-Tutorial-3.pdf']
>>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>>> VenueClientController.py:848 DEBUG Serverpath:
>>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>>> VenueClientController.py:857 DEBUG Have args, creating thread,
>>>>> url:
>>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>>> , files: [u'/home/ercea/Descargas/DSD-Tutorial-3.pdf']
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:944 DEBUG Upload: getting identity
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:949 DEBUG Got identity <?xml
>>>>> version="1.0" ? ><Subject><Subject auth_data="" auth_type="x509"
>>>>> name="/O=Access Grid/O=Argonne National Laboratory/OU=Futures Lab
>>>>> Anonymous Authority/CN=Anonymous User
>>>>> 17130398eef9fd82cba8dc9d845a1768"/></ Subject>
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:950 DEBUG get_ident_and_upload: Upload
>>>>> URL
>>>>> ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>>> ftps:
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>>> localhost:8006
>>>>> 04/22/10 21:17:00 140471479552272 VenueClientController
>>>>> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
>>>>> 7f00010133381d7afaba1c148fcabe1007
>>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1267 INFO
>>>>> UploadFiles: ftps://localhost:
>>>>> 8006/7f00010133381d7afaba1c148fcabe1007 [u'/home/ercea/Descargas/
>>>>> DSD- Tutorial-3.pdf']
>>>>> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1271
>>>>> DEBUG
>>>>> UploadFiles: ftps://localhost:
>>>>> 8006/7f00010133381d7afaba1c148fcabe1007 /home/ercea/Descargas/DSD-
>>>>> Tutorial-3.pdf [u'/home/ercea/Descargas/ DSD-Tutorial-3.pdf']
>>>>> 04/22/10 21:17:00 140471479552272 FTPSClient FTPSClient.py:141
>>>>> DEBUG
>>>>> Entered FTPSUploadFile: localfile=/home/ercea/Descargas/DSD-
>>>>> Tutorial-3.pdf url=ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
>>>>> 04/22/10 21:17:00 140471913264880 VenueClientController
>>>>> VenueClientController.py:862 DEBUG Started thread
>>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:605
>>>>> DEBUG Calling Heartbeat, time now: 1271963840
>>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:629
>>>>> DEBUG Next Heartbeat needed within 36s
>>>>> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:639
>>>>> DEBUG heartBeatCounter = 2
>>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:605
>>>>> DEBUG Calling Heartbeat, time now: 1271963876
>>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:629
>>>>> DEBUG Next Heartbeat needed within 36s
>>>>> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:639
>>>>> DEBUG heartBeatCounter = 3
>>>>>
>>>>>
>>>>> Here increment heartBeatCounter +1.
>>>>>
>>>>> Ah, I have this problem, in the same network, on ubuntu system
>>>>> (Ubuntu 9.10 64 bits), but I've tested on Windows 7 and I can
>>>>> upload
>>>>> files without problems.
>>>>>
>>>>> Regards!
>>>>>
>>>>>
>>>>> El dia 22 abr 2010 18:29, Thomas Uram <turam at mcs.anl.gov>
>>>>> escribió:
>>>>> Hi Jesus:
>>>>>
>>>>> There is probably some indication of the problem in the
>>>>> VenueClient.log file. If you submit a bug report from the Help
>>>>> menu
>>>>> after this happens, a recent portion of the VenueClient.log file
>>>>> will be included with the report, and we can review it. If you
>>>>> want
>>>>> to look yourself, the log file resides in the following locations
>>>>> based on platform:
>>>>> MacOS, Linux: ~/.AccessGrid3/Logs
>>>>> Windows: c:\documents and settings\username\application data
>>>>> \AccessGrid3\Logs (adjusted for your locale, of course)
>>>>> Tom
>>>>>
>>>>> On Apr 22, 2010, at 11:11 AM, Jesus Cea Oliva wrote:
>>>>>
>>>>> Hi list again, sorry for my constants problems xD.
>>>>>
>>>>> I'm trying upload files to our VenueServer and my problem is
>>>>> strange. Firstly, I've created a VenueServer locally, for testing.
>>>>>
>>>>> Well, when I try to upload a file, I can see the message on the
>>>>> VenueClient "Uploading file X" and a progressbar, but, when the
>>>>> progressbar is near to the end, stops and list of files (Data
>>>>> section) don't refresh. So I beleived that the file can't upload.
>>>>>
>>>>> But, really, the file has uploaded, because I can see the folder
>>>>> Data of my local VenueServer, and the file is copied in this
>>>>> folder.
>>>>> In this case, if I reset VenueServer and VenueClient, then, I can
>>>>> see the files in Data section.
>>>>>
>>>>> So I think that the file uploads fine but, the list of files don't
>>>>> refresh and the file uploaded don't appears.
>>>>>
>>>>> I must say that, sometimes, I can upload a file so I don't know
>>>>> whats really the problem (ports closed? firewall? path or name
>>>>> files? file size?) .
>>>>>
>>>>> Thanks again and apologize for any inconveniences
>>>>>
>>>>> Regards!
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> <VenueClient.log>
>>>>>
>>>>> Christoph Willing +61 7 3365 8316
>>>>> QCIF Access Grid Manager
>>>>> University of Queensland
>>>>>
>>>>> Christoph Willing +61 7 3365 8316
>>>>> QCIF Access Grid Manager
>>>>> University of Queensland
>>>>>
>>>>>
>>>>
>>>> Christoph Willing +61 7 3365 8316
>>>> QCIF Access Grid Manager
>>>> University of Queensland
>>>>
>>>
>>
>> Christoph Willing +61 7 3365 8316
>> QCIF Access Grid Manager
>> University of Queensland
>>
> <USD Venue Server log.txt>
Christoph Willing +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland
More information about the ag-tech
mailing list