[AG-TECH] Problems uploading files

Sun Apr 25 05:11:19 CDT 2010

On 23/04/2010, at 6:24 PM, Jesus Cea Oliva wrote:

> Hi Crhistoph, thanks for your response :)
>
> I try upload files since 1 o 2 K (source codes, txts, etc.) and  
> files about 3-6 Mb (PDFs, ppts...) and, all cases I have the same  
> problem. The strange of this problem is that ocurrs randomly, and  
> sometimes, the same file that before appears not upload, now upload  
> (although, the majority cases, don't upload)
>

Are you using the same venue server in each test case (Ubuntu &  
Windows VenueClient)?

Are you using your own server or a "well known" server?
If these tests are with you own server, could you instead use the APAG  
venue server at https://vv3.ap-accessgrid.org:8000/Venues/default and  
then enter the Test venue. There are several files in that venue;  
could you try to download the simple.py and methanol.obj files from  
that venue and rename them e.g. simple.py -> simple.pyX and  
methanol.obj -> methanol.objX. Now upload te renamed files back into  
the Test venue. Do they both upload as expected?

FYI, although I thought I had replicated the problem, I now find that  
it actually works correctly. My first hurried tests only appeared to  
replicate your symptoms - because I didn't wait long enogh for the  
upload to complete. However I have now performed the tests again and  
all files upload correctly - and the venue client interface shows the  
new data correctly in all cases.

I believe the previous test appeared not to update the venue client  
interface because I was performing the tests from home using an ADSL  
connection - therefore the initial download was quite fast. The upload  
was slower because of the much lower upstream bandwidth of my ADSL  
connection. My guess is that the first portion of upload _appears_  
fast because the venue client is just filling up the local TCP system  
buffer. However that buffer then has to be transferred (via slow ADSL  
uplink) to the venue server - which manifests as a fast intitial  
movement of the progress bar that then appears to freeze. The "freeze"  
is really just a pause while contents of the TCP buffer is being  
transferred across the network.

In my case the simple.py file, being small, transfers very quickly.  
The methanol.obj file is about 11M - I guess this is just a bit larger  
than my system network buffer size - and takes a lot longer to transfer.

Based on redoing the tests (using the APAG veneu server), I cannot  
replicate the problem.

Could you confirm whether you still have the problem when you use the  
APAG venue server?

chris

> On Windows I try similar files (somes files are the same), only  
> don't work once due to a very big file. As I saw that I can upload  
> files without problems... I try all file types. So I upload a file  
> about 100Mb and don't work xD.
>
> Thanks for your help :). Regards!
>
> El dia 23 abr 2010 02:35, Christoph Willing <c.willing at uq.edu.au>  
> escribió:
> I wonder if this is related to the size of the data object. I just  
> tried (with Ubuntu 9.10 64bit) and found that a small file (6K)  
> there was no problem. With a larger file (12M) I saw exactly the  
> same problem - upload appears not to complete but has actually  
> succeeded. Re-entering the venue shows that the new data is there.
>
> In your Windows test, are you uploading the same file (or something  
> of similar size) as in the Ubuntu case?
>
> I will now be away for a few days so I can't do any more  
> investigation myself until I return. Maybe others can look into this  
> in the meantime ...
>
>
> chris
>
>
> On 23/04/2010, at 5:28 AM, Jesus Cea Oliva wrote:
>
> Thanks Tom for your fast response :).
>
> Well, atach the VenueClient log in this email, I can't see any  
> problems :S.
>
>
>
> 04/22/10 21:16:50 140471913264880 VenueClient VenueClientUI.py: 1308  
> DEBUG VenueClientUI.AddDataCB: Trying to upload to 'ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007'
> 04/22/10 21:17:00 140471913264880 VenueClient VenueClientUI.py: 1395  
> DEBUG AddDataCB: URI of parent is
> 04/22/10 21:17:00 140471913264880 VenueClientController  
> VenueClientController.py:821 DEBUG In  
> VenueClientController.UploadVenueFiles
> 04/22/10 21:17:00 140471913264880 VenueClientController  
> VenueClientController.py:822 DEBUG fileList = [u'/home/ercea/  
> Descargas/DSD-Tutorial-3.pdf']
> 04/22/10 21:17:00 140471913264880 VenueClientController  
> VenueClientController.py:848 DEBUG Serverpath:
> 04/22/10 21:17:00 140471913264880 VenueClientController  
> VenueClientController.py:857 DEBUG Have args, creating thread, url: ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007 
>  , files: [u'/home/ercea/Descargas/DSD-Tutorial-3.pdf']
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:944 DEBUG Upload: getting identity
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:949 DEBUG Got identity <?xml  
> version="1.0" ? ><Subject><Subject auth_data="" auth_type="x509"  
> name="/O=Access Grid/O=Argonne National Laboratory/OU=Futures Lab  
> Anonymous Authority/CN=Anonymous User  
> 17130398eef9fd82cba8dc9d845a1768"/></ Subject>
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:950 DEBUG get_ident_and_upload: Upload URL ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:  
> ftps:
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:  
> localhost:8006
> 04/22/10 21:17:00 140471479552272 VenueClientController  
> VenueClientController.py:953 DEBUG Get_ident_and_upload: Word is:  
> 7f00010133381d7afaba1c148fcabe1007
> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1267 INFO  
> UploadFiles: ftps://localhost: 
> 8006/7f00010133381d7afaba1c148fcabe1007 [u'/home/ercea/Descargas/DSD- 
> Tutorial-3.pdf']
> 04/22/10 21:17:00 140471479552272 DataStore DataStore.py:1271 DEBUG  
> UploadFiles: ftps://localhost: 
> 8006/7f00010133381d7afaba1c148fcabe1007 /home/ercea/Descargas/DSD- 
> Tutorial-3.pdf [u'/home/ercea/Descargas/ DSD-Tutorial-3.pdf']
> 04/22/10 21:17:00 140471479552272 FTPSClient FTPSClient.py:141 DEBUG  
> Entered FTPSUploadFile: localfile=/home/ercea/Descargas/DSD-  
> Tutorial-3.pdf url=ftps://localhost:8006/7f00010133381d7afaba1c148fcabe1007
> 04/22/10 21:17:00 140471913264880 VenueClientController  
> VenueClientController.py:862 DEBUG Started thread
> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:605  
> DEBUG Calling Heartbeat, time now: 1271963840
> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:629  
> DEBUG Next Heartbeat needed within 36s
> 04/22/10 21:17:20 140471517653264 VenueClient VenueClient.py:639  
> DEBUG heartBeatCounter = 2
> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:605  
> DEBUG Calling Heartbeat, time now: 1271963876
> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:629  
> DEBUG Next Heartbeat needed within 36s
> 04/22/10 21:17:56 140471401707792 VenueClient VenueClient.py:639  
> DEBUG heartBeatCounter = 3
>
>
> Here increment heartBeatCounter +1.
>
> Ah, I have this problem, in the same network, on ubuntu system  
> (Ubuntu 9.10 64 bits), but I've tested on Windows 7 and I can upload  
> files without problems.
>
> Regards!
>
>
> El dia 22 abr 2010 18:29, Thomas Uram <turam at mcs.anl.gov> escribió:
> Hi Jesus:
>
> There is probably some indication of the problem in the  
> VenueClient.log file. If you submit a bug report from the Help menu  
> after this happens, a recent portion of the VenueClient.log file  
> will be included with the report, and we can review it. If you want  
> to look yourself, the log file resides in the following locations  
> based on platform:
> MacOS, Linux: ~/.AccessGrid3/Logs
> Windows: c:\documents and settings\username\application data  
> \AccessGrid3\Logs (adjusted for your locale, of course)
> Tom
>
> On Apr 22, 2010, at 11:11 AM, Jesus Cea Oliva wrote:
>
> Hi list again, sorry for my constants problems xD.
>
> I'm trying upload files to our VenueServer and my problem is  
> strange. Firstly, I've created a VenueServer locally, for testing.
>
> Well, when I try to upload a file, I can see the message on the  
> VenueClient "Uploading file X" and a progressbar, but, when the  
> progressbar is near to the end, stops and list of files (Data  
> section) don't refresh. So I beleived that the file can't upload.
>
> But, really, the file has uploaded, because I can see the folder  
> Data of my local VenueServer, and the file is copied in this folder.  
> In this case, if I reset VenueServer and VenueClient, then, I can  
> see the files in Data section.
>
> So I think that the file uploads fine but, the list of files don't  
> refresh and the file uploaded don't appears.
>
> I must say that, sometimes, I can upload a file so I don't know  
> whats really the problem (ports closed? firewall? path or name  
> files? file size?) .
>
> Thanks again and apologize for any inconveniences
>
> Regards!
>
>
>
>
>
>
>
>
>
> <VenueClient.log>
>
> Christoph Willing +61 7 3365 8316
> QCIF Access Grid Manager
> University of Queensland
>
>

Christoph Willing                       +61 7 3365 8316
QCIF Access Grid Manager
University of Queensland