(02:01:53 PM) jorgenpt: kohsuke: How are JNLP slaves connected to the master?
(02:02:09 PM) kohsuke: by making a TCP/IP connection
(02:02:21 PM) jorgenpt: kohsuke: I'm seeing pretty crappy performance when copying artifacts to a JNLP slave, but on the same machine, copying artifacts to the master from an SSH slave seems very fast.
(02:03:00 PM) kohsuke: it's possible that it fell back to HTTP connection
(02:03:06 PM) kohsuke: that can make things slower
(02:03:33 PM) jorgenpt: kohsuke: How can I tell?
(02:03:36 PM) jorgenpt: Nothing in the slave log
(02:04:03 PM) kohsuke: Yeah, I should log it
(02:04:09 PM) kohsuke: in the mean time, server thread dump can tell
(02:04:34 PM) kohsuke: or maybe there's some lack of buffering somewhere with TCP/IP connection of JNLP slaves
(02:04:38 PM) jorgenpt: What does that mean?
(02:04:50 PM) kohsuke: that means give me the server thread dump and I can tell you
(02:04:56 PM) kohsuke: http://yourjenkinsserver/threadDump/
(02:05:20 PM) jorgenpt: Ah, cool
(02:07:26 PM) jorgenpt: Jesus, gist.github.com is silly slow
(02:07:53 PM) jorgenpt: kohsuke: http://pastebin.com/LaszVA11
(02:08:45 PM) kohsuke: I mean the server VM not the slave VM
(02:08:50 PM) mode (+o hare_brain) by ChanServ
(02:09:29 PM) jorgenpt: Ah
(02:13:14 PM) jorgenpt: kohsuke: http://pastebin.com/ydn8J8Tw
(02:15:59 PM) kohsuke: I don't see any JNLP slaves connected here
(02:16:01 PM) kohsuke: what's the name of it?
(02:16:34 PM) jorgenpt: osx-build_gui
(02:16:58 PM) kohsuke: OK, I missed that. It's connected via TCP/IP so it's on the fast path
(02:17:03 PM) jorgenpt: Ok
(02:17:35 PM) kohsuke: that probably means there's some stupid things going on with this connection mode
(02:17:54 PM) kohsuke: maybe this is why other people often complain about the artifact copy speed
(02:18:04 PM) kohsuke: maybe it's only on certain connection mode
(02:18:09 PM) jorgenpt: Where's the code that's responsible for setting up the socket on the slave?
(02:18:31 PM) jorgenpt: It wasn't a problem until the builder and the server were at different sites
(02:18:43 PM) jorgenpt: Using iperf I still get 80-90mbit/s from the slave to the master
(02:18:57 PM) kohsuke: hudson.remoting.Launcher.runAsTcpClient()
(02:18:59 PM) jorgenpt: But Jenkins with JNLP only gets a few hundred kilobyte/s copying
(02:19:30 PM) kohsuke: Oh, don't I need TCP_NODELAY?
(02:19:41 PM) jorgenpt: In https://github.com/jenkinsci/remoting?
(02:19:46 PM) kohsuke: yes
(02:20:19 PM) jorgenpt: I don't think that should impact it when we're transmitting ~40MB
(02:20:32 PM) jorgenpt: I think TCP_NODELAY makes a difference when you're only sending a little bit of data
(02:20:39 PM) jorgenpt: (http://www.unixguide.net/network/socketfaq/2.16.shtml)
(02:21:44 PM) mkoch: hello, is https://wiki.jenkins-ci.org/display/JENKINS/Building+Jenkins up-to-date? I ask becuase I just built jenkins from source and after "mvn hudson-dev:run" I get only a 404 on http://localhost:8080
(02:22:05 PM) kohsuke: That 40MB stuff gets chopped up into a chunk by the remoting layer with TCP-like windowing and other stuff
(02:22:26 PM) kohsuke: It doesn't translate into 40MB write syscall
(02:22:45 PM) kohsuke: I think it's worth a try especially if you are willing to try it out
(02:23:08 PM) kohsuke: otherwise I can't think of anything obvious.
(02:23:20 PM) kohsuke: both server & client are buffering as can be seen in the stack trace
(02:23:28 PM) kohsuke: and otherwise it's a fairly plain socket connection
(02:24:16 PM) jorgenpt: Is there any way to set these things without recompiling etc?
(02:24:52 PM) kohsuke: we do need to recompile I'm afraid
(02:25:09 PM) jorgenpt: Our networking guy is looking at the tcpdump now
(02:25:18 PM) jorgenpt: Let me wait until he gets back to me to see what we can do
(02:25:22 PM) jorgenpt: I wish I didn't have to use JNLP :/
(02:25:25 PM) kohsuke: sounds good
(02:25:33 PM) jorgenpt: But I need Java running in a context where it can start GUI applications (ios-sim)
(02:27:05 PM) mkoch: What is the easiest way to test changes the CLI stuff?
(02:28:25 PM) kohsuke: mkoch: that doc should be up to date
(02:28:38 PM) kohsuke: what does "mvn hudson-dev:run" report?
(02:28:58 PM) mkoch: it starts the jetty server as expected
(02:30:01 PM) mkoch: I built the WAR with mvn -Plight-tests clean install
(02:30:20 PM) mkoch: before running hudson-dev:run
(02:30:32 PM) kohsuke: and no error message reported of any kind?
(02:30:44 PM) kohsuke: maybe you are running "hudson-dev:run" from top, not from war/ ?
(02:30:45 PM) mkoch: so I assume I followed everyting in the docs
(02:30:48 PM) jorgenpt: Ho, hum.
(02:30:53 PM) jorgenpt: I guess I don't actually need JNLP.
(02:31:05 PM) jorgenpt: I just need the computer to have been logged in once.
(02:32:38 PM) mkoch: gah, mistyped "cd war" and didnt recognized. Thanks for the hint.
(02:32:52 PM) mkoch: now I can finish that patch
(02:33:32 PM) kohsuke: I should have "mvn hudson-dev:run" on top fail more gracefully
(02:33:37 PM) kohsuke: I've made that mistake a number of times myself
(02:33:53 PM) kohsuke: jorgenpt: our fixing the performance problem for JNLP slaves help other users too
(02:34:08 PM) kohsuke: from my PoV, it'd be great to get this solved properly than you work around it
(02:34:18 PM) kohsuke: (and yes, I know that's not the goal you have)
(02:34:23 PM) jorgenpt: :-)
(02:34:46 PM) jorgenpt: I don't really like JNLP to begin with, because it keeps failing when the SSL certs change or we move the box :p
(02:34:51 PM) jorgenpt: Too fragile
(02:37:19 PM) mode (+v olamy) by ChanServ
(02:37:27 PM) mode (+v buep) by ChanServ
(02:38:13 PM) jorgenpt: kohsuke: Hm, I'm not sure it's JNLP related
(02:38:20 PM) jorgenpt: It seems more like it's related to the direction of the traffic
(02:38:25 PM) jorgenpt: It could be specific to our network
(02:38:32 PM) kohsuke: ok
(02:38:34 PM) jorgenpt: Copying *from* master = fast, copying *to* master = slow.
(02:38:39 PM) jorgenpt: Err
(02:38:41 PM) jorgenpt: Reverse that
(02:38:46 PM) jorgenpt: Copying *from* master is the slow one.
(02:40:37 PM) jorgenpt: Maybe it's related to the fingerprinting stuff?
(02:41:19 PM) kohsuke: that's not impossible, but I find it difficult to believe that md5 computation is slower than network transfer
(02:42:57 PM) jorgenpt: Yeah~
(02:43:06 PM) jorgenpt: 3 minutes to copy 40MB over 100mbit
(02:45:40 PM) kohsuke: if I build a binary for you, would you be willing to try?
(02:45:58 PM) kohsuke: jorgenpt:  I can create a new binary for you quickly, but for me to properly set up a test environment would take far longer
(02:46:18 PM) jorgenpt: What does the new binary do?
(02:46:23 PM) jorgenpt: If it's slave-only, then yes
(02:46:28 PM) jorgenpt: If it requires restarting the master, then no.
(02:46:39 PM) kohsuke: the latter, because TCP_NODELAY needs to be applied on both ends
(02:46:54 PM) kohsuke: all right, what if I put this in now, and we come back to this in the next update?
(02:47:19 PM) jorgenpt: Sure
(02:49:15 PM) kohsuke: any other socket options that I could be using while we are at it?
(02:50:08 PM) jorgenpt: No idea :)
(02:50:21 PM) kohsuke: looking at jetty
(02:50:30 PM) kohsuke: it's using nodelay, so_timeout and soLinger
(02:50:39 PM) jorgenpt: Hrm, I guess we updated Jenkins a bunch of versions at the same time as moving it to a different site
(02:50:47 PM) jorgenpt: So it could be related to Jenkins versions too :/
(02:51:21 PM) kohsuke: I don't think I need SO_LINGER

https://ci.jenkins-ci.org/job/jenkins_rc_branch/285/testReport/hudson.tasks.junit/SuiteResultTest/sizeSurefire/ failed at 4 over.

Probably due to some as-yet-undiagnosed interaction between variable workdir path lengths and 8-byte alignment boundaries.
(Could not reproduce on Linux after running from various long workdirs.)

[FIXED JENKINS-10569] When installing plugins with overlapping dependencies, Jenkins downloads the duplicate plugins multiple times

[FIXED JENKINS-15540] Update logging levels in LogRotator

while I can't pinpoint what's really causing this, moving this
dependency to core seems to solve the problem.

Guava is defined as dependency in org.kohsuke.stapler:stapler, then
redefined to a newer version in maven-plugin. The latter gets to plugins
as a test scope. When I look at the dependency scope of this in a plugin
project in IntelliJ IDEA, it reports this dependency as the test scope
(and this makes the error somewhat understandable.)

[FIX JENKINS-10569] When installing plugins with overlapping dependencies, Jenkins downloads the duplicate plugins multiple times

refactor /manage to use Management links

execScript cannot return value [1], so I'm restricting its use to where
we don't care about the return value. See ticket for more discussions

[1] http://msdn.microsoft.com/en-us/library/ie/ms536420(v=vs.85).aspx

Does not matter for any handlers used in Jenkins, but can break third-party frameworks.
In particular mixpanel-2.1.min.js seems to detect when this mistake is made and somehow disables some Jenkins handlers,
causing e.g. “LOADING” to stay on job config pages and breaking behavior.js (but only on Firefox).

Using addEventListener/attachEvent where available also works, but this fix is a bit simpler and more general.
https://gist.github.com/1930440 would also work but again seems like overkill here.

To reproduce:
diff --git a/core/src/main/resources/lib/layout/layout.jelly b/core/src/main/resources/lib/layout/layout.jelly
index 1fb3fec..c1337c4 100644
--- a/core/src/main/resources/lib/layout/layout.jelly
+++ b/core/src/main/resources/lib/layout/layout.jelly
@@ -88,6 +88,19 @@ ${h.initPageVariables(context)}
 <x:doctype name="html" />
 <html>
   <head>
+<script type="text/javascript"><![CDATA[
+(function(c,a){window.mixpanel=a;var b,d,h,e;b=c.createElement("script");
+    b.type="text/javascript";b.async=!0;b.src=("https:"===c.location.protocol?"https:":"http:")+
+    '//cdn.mxpnl.com/libs/mixpanel-2.1.min.js';d=c.getElementsByTagName("script")[0];
+    d.parentNode.insertBefore(b,d);a._i=[];a.init=function(b,c,f){function d(a,b){
+    var c=b.split(".");2==c.length&&(a=a[c[0]],b=c[1]);a[b]=function(){a.push([b].concat(
+    Array.prototype.slice.call(arguments,0)))}}var g=a;"undefined"!==typeof f?g=a[f]=[]:
+    f="mixpanel";g.people=g.people||[];h=['disable','track','track_pageview','track_links',
+    'track_forms','register','register_once','unregister','identify','name_tag',
+    'set_config','people.identify','people.set','people.increment'];for(e=0;e<h.length;e++)d(g,h[e]);
+    a._i.push([b,c,f])};a.__SV=1.1;})(document,window.mixpanel||[]);
+    mixpanel.init("…something valid here…");
+]]></script>
     ${h.checkPermission(it,permission)}

     <title>${h.appendIfNotNull(title, ' [Jenkins]', 'Jenkins')}</title>

Integrated a fix made in Stapler.

No need for a special logging level flag; if you are interested in this logger, just set its level to FINE.

When calculating idle executors, only include Computers which are accepting tasks.

[FIXED JENKINS-15418] Allow long paths on windows.

Reduce the noise caused by MavenArtifactArchiver when Maven artifact archiving is disabled.
Thank

archiving is disabled.

Only display one line per execution instead of displaying one line per
non-archived file.

This needs to be eval, not geval.
Plus I forgot to return the value.

Evaluate script in global context.
See http://perfectionkills.com/global-eval-what-are-the-options/ for how
to evaluate JavaScript in global context.