[AG-TECH] AG3 Bridge Registry Down

[Andrew Sharpe]

Hi Tom, I'm having problems with the bridge registry right now.

The VenueClient does start eventually, but there are no bridges available.

BTW, for those using the RPMs from UQ, you can get the client to start quicker by running it it "debug mode" and pressing ctrl+c when it hangs at the bridge registry step.

Here's an excerpt from VenueClient.log

01/10/2007 02:16:54 PM -1208637776 VenueClient     VenueClient.py:242 DEBUG get bridges from registry
01/10/2007 02:22:03 PM -1208637776 RegistryClient     RegistryClient.py:46 ERROR Failed to connect to registry vv3.mcs.anl.gov:8030
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/AccessGrid3/AccessGrid/Registry/RegistryClient.py", line 41, in _connectToRegistry
    if self.PingRegistryPeer(tmpServerProxy) > -1:
  File "/usr/lib/python2.4/site-packages/AccessGrid3/AccessGrid/Registry/RegistryClient.py", line 58, in PingRegistryPeer
    startTime = serverProxy.Ping(time.time())
  File "/usr/lib/python2.4/xmlrpclib.py", line 1096, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.4/xmlrpclib.py", line 1383, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.4/xmlrpclib.py", line 1131, in request
    errcode, errmsg, headers = h.getreply()
  File "/usr/lib/python2.4/httplib.py", line 1134, in getreply
    response = self._conn.getresponse()
  File "/usr/lib/python2.4/httplib.py", line 863, in getresponse
    response.begin()
  File "/usr/lib/python2.4/httplib.py", line 333, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.4/httplib.py", line 291, in _read_status
    line = self.fp.readline()
  File "/usr/lib/python2.4/socket.py", line 325, in readline
    data = recv(1)
error: (104, 'Connection reset by peer')
01/10/2007 02:22:03 PM -1208637776 RegistryClient     RegistryClient.py:50 INFO  No bridge registry peers reachable
01/10/2007 02:22:03 PM -1208637776 VenueClient     VenueClient.py:253 ERROR __LoadBridges: Can not connect to bridge registry http://www.accessgrid.org/registry/peers.txt
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/AccessGrid3/AccessGrid/VenueClient.py", line 248, in __LoadBridges
    bridgeList = self.registryClient.LookupBridge(10)
  File "/usr/lib/python2.4/site-packages/AccessGrid3/AccessGrid/Registry/RegistryClient.py", line 109, in LookupBridge
    bridges = self.serverProxy.LookupBridge()
AttributeError: 'NoneType' object has no attribute 'LookupBridge'



Wireshark shows that the original connection is established, then the following RPC call is made:

POST /RPC2 HTTP/1.0
Host: vv3.mcs.anl.gov:8030
User-Agent: xmlrpclib.py/1.0.1 (by www.pythonware.com)
Content-Type: text/xml
Content-Length: 166

The packet that sends this information never receives an ack for it, so we get repeated retransmittion from our client, and we're receiving duplicate acks for the connection establishing packet from you.  This indicates that our RPC request is not getting to you!  

I've attached the dump for your perusal.

I'm sure this debate has been and gone but this single point of failure for bridging is uncool IMHO.

Thanks, Andrew


---- Original message ----
>Date: Thu, 04 Jan 2007 17:22:00 -0600
>From: "Thomas D. Uram" <turam at mcs.anl.gov>  
>Subject: Re: [AG-TECH] AG3 Bridge Registry Down  
>To: Chris Kendrick <kendrick at vpac.org>
>Cc: ag-tech at mcs.anl.gov
>
>I tried it just now and it worked fine for me.  If the problem
>persists, let me know.
>
>Tom
>
>
>On 1/4/07 5:16 PM, Chris Kendrick wrote:
>> Hi
>>
>> Is the AG3 bridge registry down again?  I wasn't getting a list of
>> bridges at 10:10am 5th Jan Australia, Melbourne Time  (23:10 4th Jan
>> UTC)
>>
>> Chris Kendrick
>> VPAC
>>
>>
>>   
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bridge_registry.pcap
Type: application/octet-stream
Size: 3194 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/ag-tech/attachments/20070110/8a629d3b/attachment.obj>