This allows TCP slave agent port to be reverse proxied.

The JNLP port is not interesting enough to warrant logging at INFO during every startup, including the vast majority of functional tests which do not even use JNLP slaves.

See f8916a839c5a7bfe6d0de100a33cf23de33de6e4 for the corresponding change in the remoting.

Since the error messages only use ASCII

(02:01:53 PM) jorgenpt: kohsuke: How are JNLP slaves connected to the master?
(02:02:09 PM) kohsuke: by making a TCP/IP connection
(02:02:21 PM) jorgenpt: kohsuke: I'm seeing pretty crappy performance when copying artifacts to a JNLP slave, but on the same machine, copying artifacts to the master from an SSH slave seems very fast.
(02:03:00 PM) kohsuke: it's possible that it fell back to HTTP connection
(02:03:06 PM) kohsuke: that can make things slower
(02:03:33 PM) jorgenpt: kohsuke: How can I tell?
(02:03:36 PM) jorgenpt: Nothing in the slave log
(02:04:03 PM) kohsuke: Yeah, I should log it
(02:04:09 PM) kohsuke: in the mean time, server thread dump can tell
(02:04:34 PM) kohsuke: or maybe there's some lack of buffering somewhere with TCP/IP connection of JNLP slaves
(02:04:38 PM) jorgenpt: What does that mean?
(02:04:50 PM) kohsuke: that means give me the server thread dump and I can tell you
(02:04:56 PM) kohsuke: http://yourjenkinsserver/threadDump/
(02:05:20 PM) jorgenpt: Ah, cool
(02:07:26 PM) jorgenpt: Jesus, gist.github.com is silly slow
(02:07:53 PM) jorgenpt: kohsuke: http://pastebin.com/LaszVA11
(02:08:45 PM) kohsuke: I mean the server VM not the slave VM
(02:08:50 PM) mode (+o hare_brain) by ChanServ
(02:09:29 PM) jorgenpt: Ah
(02:13:14 PM) jorgenpt: kohsuke: http://pastebin.com/ydn8J8Tw
(02:15:59 PM) kohsuke: I don't see any JNLP slaves connected here
(02:16:01 PM) kohsuke: what's the name of it?
(02:16:34 PM) jorgenpt: osx-build_gui
(02:16:58 PM) kohsuke: OK, I missed that. It's connected via TCP/IP so it's on the fast path
(02:17:03 PM) jorgenpt: Ok
(02:17:35 PM) kohsuke: that probably means there's some stupid things going on with this connection mode
(02:17:54 PM) kohsuke: maybe this is why other people often complain about the artifact copy speed
(02:18:04 PM) kohsuke: maybe it's only on certain connection mode
(02:18:09 PM) jorgenpt: Where's the code that's responsible for setting up the socket on the slave?
(02:18:31 PM) jorgenpt: It wasn't a problem until the builder and the server were at different sites
(02:18:43 PM) jorgenpt: Using iperf I still get 80-90mbit/s from the slave to the master
(02:18:57 PM) kohsuke: hudson.remoting.Launcher.runAsTcpClient()
(02:18:59 PM) jorgenpt: But Jenkins with JNLP only gets a few hundred kilobyte/s copying
(02:19:30 PM) kohsuke: Oh, don't I need TCP_NODELAY?
(02:19:41 PM) jorgenpt: In https://github.com/jenkinsci/remoting?
(02:19:46 PM) kohsuke: yes
(02:20:19 PM) jorgenpt: I don't think that should impact it when we're transmitting ~40MB
(02:20:32 PM) jorgenpt: I think TCP_NODELAY makes a difference when you're only sending a little bit of data
(02:20:39 PM) jorgenpt: (http://www.unixguide.net/network/socketfaq/2.16.shtml)
(02:21:44 PM) mkoch: hello, is https://wiki.jenkins-ci.org/display/JENKINS/Building+Jenkins up-to-date? I ask becuase I just built jenkins from source and after "mvn hudson-dev:run" I get only a 404 on http://localhost:8080
(02:22:05 PM) kohsuke: That 40MB stuff gets chopped up into a chunk by the remoting layer with TCP-like windowing and other stuff
(02:22:26 PM) kohsuke: It doesn't translate into 40MB write syscall
(02:22:45 PM) kohsuke: I think it's worth a try especially if you are willing to try it out
(02:23:08 PM) kohsuke: otherwise I can't think of anything obvious.
(02:23:20 PM) kohsuke: both server & client are buffering as can be seen in the stack trace
(02:23:28 PM) kohsuke: and otherwise it's a fairly plain socket connection
(02:24:16 PM) jorgenpt: Is there any way to set these things without recompiling etc?
(02:24:52 PM) kohsuke: we do need to recompile I'm afraid
(02:25:09 PM) jorgenpt: Our networking guy is looking at the tcpdump now
(02:25:18 PM) jorgenpt: Let me wait until he gets back to me to see what we can do
(02:25:22 PM) jorgenpt: I wish I didn't have to use JNLP :/
(02:25:25 PM) kohsuke: sounds good
(02:25:33 PM) jorgenpt: But I need Java running in a context where it can start GUI applications (ios-sim)
(02:27:05 PM) mkoch: What is the easiest way to test changes the CLI stuff?
(02:28:25 PM) kohsuke: mkoch: that doc should be up to date
(02:28:38 PM) kohsuke: what does "mvn hudson-dev:run" report?
(02:28:58 PM) mkoch: it starts the jetty server as expected
(02:30:01 PM) mkoch: I built the WAR with mvn -Plight-tests clean install
(02:30:20 PM) mkoch: before running hudson-dev:run
(02:30:32 PM) kohsuke: and no error message reported of any kind?
(02:30:44 PM) kohsuke: maybe you are running "hudson-dev:run" from top, not from war/ ?
(02:30:45 PM) mkoch: so I assume I followed everyting in the docs
(02:30:48 PM) jorgenpt: Ho, hum.
(02:30:53 PM) jorgenpt: I guess I don't actually need JNLP.
(02:31:05 PM) jorgenpt: I just need the computer to have been logged in once.
(02:32:38 PM) mkoch: gah, mistyped "cd war" and didnt recognized. Thanks for the hint.
(02:32:52 PM) mkoch: now I can finish that patch
(02:33:32 PM) kohsuke: I should have "mvn hudson-dev:run" on top fail more gracefully
(02:33:37 PM) kohsuke: I've made that mistake a number of times myself
(02:33:53 PM) kohsuke: jorgenpt: our fixing the performance problem for JNLP slaves help other users too
(02:34:08 PM) kohsuke: from my PoV, it'd be great to get this solved properly than you work around it
(02:34:18 PM) kohsuke: (and yes, I know that's not the goal you have)
(02:34:23 PM) jorgenpt: :-)
(02:34:46 PM) jorgenpt: I don't really like JNLP to begin with, because it keeps failing when the SSL certs change or we move the box :p
(02:34:51 PM) jorgenpt: Too fragile
(02:37:19 PM) mode (+v olamy) by ChanServ
(02:37:27 PM) mode (+v buep) by ChanServ
(02:38:13 PM) jorgenpt: kohsuke: Hm, I'm not sure it's JNLP related
(02:38:20 PM) jorgenpt: It seems more like it's related to the direction of the traffic
(02:38:25 PM) jorgenpt: It could be specific to our network
(02:38:32 PM) kohsuke: ok
(02:38:34 PM) jorgenpt: Copying *from* master = fast, copying *to* master = slow.
(02:38:39 PM) jorgenpt: Err
(02:38:41 PM) jorgenpt: Reverse that
(02:38:46 PM) jorgenpt: Copying *from* master is the slow one.
(02:40:37 PM) jorgenpt: Maybe it's related to the fingerprinting stuff?
(02:41:19 PM) kohsuke: that's not impossible, but I find it difficult to believe that md5 computation is slower than network transfer
(02:42:57 PM) jorgenpt: Yeah~
(02:43:06 PM) jorgenpt: 3 minutes to copy 40MB over 100mbit
(02:45:40 PM) kohsuke: if I build a binary for you, would you be willing to try?
(02:45:58 PM) kohsuke: jorgenpt:  I can create a new binary for you quickly, but for me to properly set up a test environment would take far longer
(02:46:18 PM) jorgenpt: What does the new binary do?
(02:46:23 PM) jorgenpt: If it's slave-only, then yes
(02:46:28 PM) jorgenpt: If it requires restarting the master, then no.
(02:46:39 PM) kohsuke: the latter, because TCP_NODELAY needs to be applied on both ends
(02:46:54 PM) kohsuke: all right, what if I put this in now, and we come back to this in the next update?
(02:47:19 PM) jorgenpt: Sure
(02:49:15 PM) kohsuke: any other socket options that I could be using while we are at it?
(02:50:08 PM) jorgenpt: No idea :)
(02:50:21 PM) kohsuke: looking at jetty
(02:50:30 PM) kohsuke: it's using nodelay, so_timeout and soLinger
(02:50:39 PM) jorgenpt: Hrm, I guess we updated Jenkins a bunch of versions at the same time as moving it to a different site
(02:50:47 PM) jorgenpt: So it could be related to Jenkins versions too :/
(02:51:21 PM) kohsuke: I don't think I need SO_LINGER

It uses Diffie Hellman to come up with one-time session key, then have
the server sign this session key to allow the client to verify that
there's no man in the middle.

and an implementation around SSH keys.

changelog will be updated when the server-side authentication module
is added.

Just import changes, from hudson.model.Jenkins -> jenkins.model.Jenkins,
and Javadoc comments.
Signed-off-by: NNigel Magnay <nigel.magnay@gmail.com>

Signed-off-by: NNigel Magnay <nigel.magnay@gmail.com>

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@33855 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@28502 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@23883 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@22037 71c3de6d-444a-0410-be80-ed276b4c234a

add some copyright headers, some tab->ws


git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@21654 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@19983 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@19975 71c3de6d-444a-0410-be80-ed276b4c234a

I discovered that when Apache sits in front of Hudson as a reverse proxy, it blocks the traffic in such a way that breaks tunneling of the remoting.

The issue is that the CLI initiates the upload side of the connection and sends some small chunk, yet Apache never forwads that to Hudson, presumably because it's trying to buffer more contents from CLI, which never arrives.

A similar problem seems to exist for the download side, although various reports (like http://mail-archives.apache.org/mod_mbox/httpd-users/200509.mbox/%3C20050927152742.GA8726@redhat.com%3E) suggests that this is fixed in a relatively recent version of Apache.

Other pages, like http://httpd.apache.org/docs/2.2/mod/mod_proxy.html , seems to suggest that Apache can be configured not to do this buffering, but my attempt to configure them failed, both on Ubuntu and OpenSolaris. It could be my mistake, but the lesson for me is that even if Apache can be configured not to interfere, it's still rather error prone.

On top of that, it's not hard to imagine that other forward/reverse proxy have similar problems --- IIRC, Dean Yu reported a similar problem.

So I decided to let CLI talk to Hudson via a conventional TCP/IP connection, if possible. An extra bonus is that this is more resource efficient on the server side (which was tying 2 request handling for pushing and pulling before this change.)

The downside of this change is that if the client is behind the proxy, or if the server is behind a reverse proxy, this needs an extra set up. I thought this is a lesser evil, as Hudson tends to run inside a corporate network, and this issue is already better understood since JNLP slaves have the same issue.

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@19378 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@15920 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@15081 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@12963 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@11280 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@9759 71c3de6d-444a-0410-be80-ed276b4c234a

extension points shouldn't depend on implementation detail class, so promoted ComputerImpl to SlaveComputer.

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@9402 71c3de6d-444a-0410-be80-ed276b4c234a

moved secret key from TCP agent listener to the core so that it can be used for other things (like remember-me authentication)


git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@6760 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3347 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3344 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3342 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3174 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3099 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@3097 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@2368 71c3de6d-444a-0410-be80-ed276b4c234a

git-svn-id: https://hudson.dev.java.net/svn/hudson/trunk/hudson/main@2296 71c3de6d-444a-0410-be80-ed276b4c234a