Partychapp not responding to messages

Partychapp appears online, but is not responding to messages. We're not sure why, but we're assuming it's an issue with AppEngine. Will investigate

Update: Back to normal - issues were due to http://code.google.com/status/appengine

partych.at bots fixed

We (ok, Craig) figured out why @partych.at bots were not accepting new invites -- basically we'd hit the number of un-answered invites the jabber server could buffer. Clearing this buffer appears to have fixed the invite issue.

This is probably unrelated to the dropped messages issue. I did upgrade the PartyChat server from Smack 3.0.1 to 3.1.0 last night which can only help.

If you're still seeing offline issues or dropped messages, keep letting us know (via comments on this blog is good) and we'll keep poking around till we get this thing fixed.

Update: partych.at bots may appear offline for you. This is because the partych.at XMPP server's roster database (i.e. who is friends with who) is now out-of-sync with your chat client. If you're using something like Pidgin, you can "unsubscribe" and then "re-request authorization" to the bot. If you're in a GMail based client, we're still trying to figure out how to do it.

Update 2: If you're using gmail chat (or something similar) go to the "Add Contact" dialogue and re-enter the partych.at addresses you were using. This will cause them to re-appear as online to you.

Missing bots and dropped messages

As many of you are aware, there are a number of problems with partychat right now; ranging from bots appearing offline to missing messages.

A few of us are trying diagnose what the underlying issues are. In general, these sorts of problems seem to stem from our connection to Google Talk servers which we have very little control over. Hopefully we'll get everything sorted out and fixed to the best of our ability soon.

Thanks to everyone who's been reporting problems -- it's touching to hear how many different people around the world are using partychat (and I share your frustration with how much of a disruption to your life it is when it's not working properly).

Apartment internet is out

So partychat is down too. Will bring it back up when connectivity is restored.

Update: This appears to actually be an issue with Google connectivity (PartyChat uses Google Talk as its Jabber host)

Update: Aaaaaand we're back!

so much for dual-homing

Well, we're in the process of switching service providers and Mihai's power just went out, so our "dual homing, it can't fail!" strategy is not looking so solid now. The state file is lost behind's Mihai's power failure, so rather than roll back the hands of time yet again, I've brought up tahcytrap@gmail.com (you remember tahcytrap, don't you?). I've purposely given it no state or ppblog -- it's a temporary hold-over for the pchat addicts out there. When we're able to get PartyChat running back and proper, I'll take tahcytrap back down.

Update: everything back to normal now

and again

Well things got blanked and it turns out the automated backups were corrupted (they were incomplete files, thus bad XML. I don't know why), so I had to turn the clock back to 6/26.

I wish I could promise things were going to get more stable, but I don't have much time these days to do anything but tend to pchat's fires. c'est la vie i guess

i'm sorry

The woeful tail of how I lost the last 3 months of your PartyChat room data

I run a pretty spit 'n polish establishment here at Techwalla. And by that I mean, PartyChat runs on a "dedicated" P3 900Mhz machine in my living room, which emits a high pitched whine and occasional harddrive clack, much to the discomfort of anyone trying to sleep on the couch next to it. When PartyChat has issues, I log into the machine and restart things. There are many things that can go wrong with this, unfortunately there's been little incentive for me to improve the situation. We have some plans to make things much more stable (Real Soon Now™), but they're a ways off.

Anyways, I accidentally (whoops) deleted the file that stores all of the state for each PartyChat room (mainly: what rooms exist, who's in what room, what their alias is). The last back-up I have is from April 2nd (ouch). I've gone and substituted in that file, so you'll notice that your rooms have just hoped into a time-machine back 3 months (old aliases, old membership rosters). If you created a PartyChat room after April 2nd (and this is the part I'm really sorry about) you're going to have to re-make that room.

Clearly I should have been making more regular (off-site even!) backups of the state file and you bet your bottom that's what I'm going to start doing now. But that's cold comfort... and I'm sorry.

issues w/ the federation

We're occasionally seeing issues where messages between non-Google domains (e.g. partych.at) and Google domains (e.g. gmail.com) are not being delivered (this is the "xxxx did not get your message" errors you sometimes get).
I'm sorry to say there's nothing I can really do about this but I've e-mailed the Google Talk team about it and it should resolve itself fairly quickly (hopefully permanently).

Today's issues...

As a lot of you are probably seeing, PartyChat has been very flakey today. I'm still trying to figure out what's happening, so if any of you are smack experts, I could appreciate the help.

The previously running process starts reporting:

[java] Found invalid presence mode

Upon restarting, it outputs:

[java] Trouble connecting to Google Talk

and on a few occasions:

[java] Trouble connecting to Google Talk
[java] java.net.SocketException: Broken pipe
[java] at java.net.SocketOutputStream.socketWrite0(Native Method)
[java] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
[java] at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
[java] at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
[java] at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
[java] at java.io.BufferedWriter.flush(BufferedWriter.java:236)
[java] at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:274)
[java] at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
[java] at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
[java] java.net.SocketException: Connection reset
[java] at java.net.SocketInputStream.read(SocketInputStream.java:168)
[java] at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java:411)
[java] at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java:453)
[java] at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
[java] at java.io.InputStreamReader.read(InputStreamReader.java:167)
[java] at java.io.BufferedReader.fill(BufferedReader.java:136)
[java] at java.io.BufferedReader.read1(BufferedReader.java:187)
[java] at java.io.BufferedReader.read(BufferedReader.java:261)
[java] at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:2971)
[java] at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)
[java] at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)
[java] at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
[java] at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)
[java] at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
[java] at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)

or

[java] java.net.SocketException: Socket closed
[java] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
[java] at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
[java] at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
[java] at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
[java] at java.io.BufferedWriter.flush(BufferedWriter.java:236)
[java] at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:274)
[java] at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
[java] at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
[java] javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
[java] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:739)
[java] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1025)
[java] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1038)
[java] at org.jivesoftware.smack.XMPPConnection.proceedTLSReceived(XMPPConnection.java:1127)
[java] at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:313)
[java] at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
[java] at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)
[java] Caused by: java.io.EOFException: SSL peer shut down incorrectly
[java] at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:321)
[java] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:720)
[java] ... 6 more
[java] java.net.SocketException: Connection closed by remote host
[java] at com.sun.net.ssl.internal.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1168)
[java] at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:43)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
[java] at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
[java] at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
[java] at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
[java] at java.io.BufferedWriter.flush(BufferedWriter.java:236)
[java] at org.jivesoftware.smack.PacketWriter.writePackets(PacketWriter.java:274)
[java] at org.jivesoftware.smack.PacketWriter.access$000(PacketWriter.java:40)
[java] at org.jivesoftware.smack.PacketWriter$1.run(PacketWriter.java:87)
[java] java.io.EOFException: no more data available - expected end tag to close start tag from line 1, parser stopped on END_TAG seen ...... @1:342
[java] at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:3014)
[java] at org.xmlpull.mxp1.MXParser.more(MXParser.java:3025)
[java] at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1144)
[java] at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
[java] at org.jivesoftware.smack.PacketReader.parsePackets(PacketReader.java:368)
[java] at org.jivesoftware.smack.PacketReader.access$000(PacketReader.java:44)
[java] at org.jivesoftware.smack.PacketReader$1.run(PacketReader.java:76)

While getting these errors, I was able to log partychat@ in to Google Talk via GMail Chat. Running PartyChat on other machines has a little more success, but not really -- half the accounts still die, but a few are able to connect.

Before I chalk this up to some sort of weird rate-limiting issue, I'm hoping someone can give me a more plausible explanation of what be happening. Any ideas?