Client-side moxi falls over (hung connections)

Couchbase Server
1

i am having trouble with client-side moxi.

it chugs along for a while with no problems, then all of a sudden i get a few of my client-to-moxi connections that hang.

the percentage of connections that hang is fairly stable (tho after a reboot that number may be slightly different), and results in an error/timeout rate of about the same percentage.

this is using the default config values, with a transaction rate of 2.5k to 4k requests per second.

there are three downstream couchabase nodes moxi is connecting to.

any idea?

thanks.

2

Some thoughs…

Perhaps try cranking up the logging to look for suspicious log messages.

And/or, perhaps telnet into moxi (like port 1121) and do a “stats proxy” command. I usually then I crawl through the output looking error counts for any root cause smoking guns.

3

thanks for the suggestions.

tried those, no luck.

btw, the moxi logging levels can be describe as such:

v == very concise amount of not-helpful
vv == medium amount of still-not-helpful
vvv == gigabytes per minute of can-you-believe-its-still-not-helpful

;o)

4

Changing the memcached client’s failure mode to Cancel (from Redistribute) seems to have fixed the problem.