Turbostat neglects to display all package C-states for some Skylake Xeon BIOS configurations.

This is due to a typo in the table decoding MSR_PKG_CST_CONFIG_CONTROL (0x000000e2)

Here we fix that typo, according to Intel SDM, vol 4, Table 2-41 -
"MSRs Supported by Intel® Xeon® Processor Scalable Family with DisplayFamily_DisplayModel 06_55H".
Signed-off-by: NArtem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

This reverts commit c91fc851.

That change caused a C6 and PC6 residency regression on large idle systems.

Users also complained about new output indicating jitter:

turbostat: cpu6 jitter 3794 9142
Signed-off-by: NLen Brown <len.brown@intel.com>
Cc: 4.13+ <stable@vger.kernel.org> # v4.13+
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>

otherwise, turbostat bails on on AMD Opteron boxes:

turbostat: cpu26: msr offset 0x1a0 read failed: Input/output error
Reported-by: NKamil Kolakowski <kkolakow@redhat.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

Turbostat has the capability to set its own affinity to
each CPU so that its MSR accesses are on the local CPU.

However, using the in-kernel cross-call in  the msr driver
tends to be less invasive, so do that -- by-default.
'-m' remains to get the old behaviour.
Signed-off-by: NLen Brown <len.brown@intel.com>

The --debug option now pre-pends each row with
the number  of micro-seconds [usec] to collect
the finishing snapshot for that row.
Signed-off-by: NLen Brown <len.brown@intel.com>

Skylake has some new counters, and they were erroneously
exempt  from --show and --hide

eg.

turbostat  --quiet --show CPU
CPU	Totl%C0	Any%C0	GFX%C0	CPUGFX%
-	116.73	90.56	85.69	79.00
0	117.78	91.38	86.47	79.71
2
1
3

is now

CPU
-
0
2
1
3
Signed-off-by: NLen Brown <len.brown@intel.com>

Signed-off-by: NLen Brown <len.brown@intel.com>

Most CPUs do not have a hardware c1 counter,
and so turbostat derives c1 residency:

c1 = TSC - MPERF - other_core_cstate_counters

As it is not possible to atomically read these coutners,
measurement jitter can case this calcuation to "go negative"
when very close to 0.  Turbostat detect that case and
simply prints c1 = 0.00%

But that check neglected to account for systems where the TSC
crystal clock domain and the MPERF BCLK domain are differ by
a small amount.  That allowed very small negative c1 numbers
to escape this check and be printed as huge positve numbers.

This code begs for a bit of cleanup, but this patch
is the minimal change to fix the issue.
Signed-off-by: NLen Brown <len.brown@intel.com>

Syntax only.

The HWP CAPABILTIES and REQUEST ratios are more easily
viewed in decimal -- just multiply by 100 and you get MHz...

new:
cpu0: MSR_HWP_CAPABILITIES: 0x010c1b23 (high 35 guar 27 eff 12 low 1)
cpu0: MSR_HWP_REQUEST: 0x80002301 (min 1 max 35 des 0 epp 0x80 window 0x0 pkg 0x0)

old:
cpu0: MSR_HWP_CAPABILITIES: 0x010c1b23 (high 0x23 guar 0x1b eff 0xc low 0x1)
cpu0: MSR_HWP_REQUEST: 0x80002301 (min 0x1 max 0x23 des 0x0 epp 0x80 window 0x0 pkg 0x0)
Signed-off-by: NLen Brown <len.brown@intel.com>

cpu0: MSR_IA32_TEMPERATURE_TARGET: 0x00641400 (100 C)
cpu0: MSR_IA32_PACKAGE_THERM_STATUS: 0x884b0800 (25 C)
cpu0: MSR_IA32_PACKAGE_THERM_INTERRUPT: 0x00000003 (100 C, 100 C)

Enable the same per-core output, but hide it behind --debug
because it is too verbose on big systems.
Signed-off-by: NLen Brown <len.brown@intel.com>

While the current SDM is silent on the matter, the Core and GFX
RAPL power meters on SKL and KBL appear to work -- so show them.
Reported-by: NYaroslav Isakov <yaroslav.isakov@gmail.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

turbostat displays a GFXMHz column, which comes from reading
/sys/class/graphics/fb0/device/drm/card0/gt_cur_freq_mhz

But GFXMHz was not changing, even when a manual
cat /sys/class/graphics/fb0/device/drm/card0/gt_cur_freq_mhz
showed a new value.

It turns out that a rewind() on the open file is not sufficient,
fflush() (or a close/open) is needed to read fresh values.
Reported-by: NYaroslav Isakov <yaroslav.isakov@gmail.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

The turbostat before this last set of changes is obsolete.
This new version can do a lot more, but it also has
some different defaults, that might catch some off-guard.
So it seems a good time to give a new version number.
Signed-off-by: NLen Brown <len.brown@intel.com>

When the "u32" keyword is used with --add, it means that
the output should be truncated to 32-bits.  This was not
happening and all 64-bits were printed.

Also, when no column name was used for an added MSR,
The default column name was in deximal, eg. MSR16.
Users report that they tend to use hex MSR numbers,
so print them in hex.  To always fit into the columns,
use the syntax M0x10.  Note that the user can always
supply any column header that they want.

eg --add msr0x10,MY_TSC
Signed-off-by: NLen Brown <len.brown@intel.com>

When turbostat is run in one-shot command mode,
the parent takes the 'before' counter snapshot,
fork/exec/wait for the child to exit,
takes the 'after' counter snapshot,
and prints the results.

however, if the child fails to exec the command,
it immediately returns, without indicating that
anythign was wrong.

Add an error message showing that exec failed:

sudo turbostat sleeeep 4
...
turbostat: exec sleeeep: No such file or directory
...

Note that the parent will still print out the statistics,
because it can't tell the difference between the failed
exec and a command that is purposefully returning
the same status.  Unfortunately, this may obscure the
error message.  However, if the --out parameter is used,
the error message is evident on stderr.
Reported-by: NWendy Wang <wendy.wang@intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

cpu1: cpufreq driver: acpi-cpufreq
cpu1: cpufreq governor: ondemand
cpufreq boost: 1

or

cpu0: cpufreq driver: intel_pstate
cpu0: cpufreq governor: powersave
cpufreq intel_pstate no_turbo: 0
Signed-off-by: NLen Brown <len.brown@intel.com>

On multi-package systems, the "Package" column was being displayed
only if --debug was used.  Show it always.
Signed-off-by: NLen Brown <len.brown@intel.com>

Originally, the only way to hide the sysfs C-state statistics columns
was with "--hide sysfs".  This was because we process "--hide" before
we probe for those columns.

hack --hide to remember deferred hide requests, and apply
them when sysfs is probed.

"--hide sysfs" is still available as short-hand to refer to
the entire group of counters.

The down-side of this change is that we no longer error check for
bogus --hide column names.  But the user will quickly figure that
out if a column they mean to hide is still there...
Signed-off-by: NLen Brown <len.brown@intel.com>

--Package is now "--cpu package",
which will display just the 1st CPU in each package

--processor is not "--cpu core"
which will display just the 1st CPU in each core
Signed-off-by: NLen Brown <len.brown@intel.com>

Make it possible to take the entire un-edited output
from `turbostat --list` and feed it to "turbostat --show"
or "turbostat --hide".

To do this, the leading comma was removed
(no mater what columns are active)
and also they dynamic C-state "C1, C2, C3" etc are replaced
by the string "sysfs", which refers to them as a group.
Signed-off-by: NLen Brown <len.brown@intel.com>

When a counter overlfows 7 columns, it shifts the remaining
columns to the right, so they no longer line up under
their column header.

Update turbostat to dectect when it is handling large
numbers, and switch to wider columns where, necessary.
Reported-by: NArtem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

It is handy to know the list of column header names,
so that they can be used with --add and --skip

The new --list option shows them:

sudo ./turbostat --list --hide sysfs
,Core,CPU,Avg_MHz,Busy%,Bzy_MHz,TSC_MHz,IRQ,SMI,CPU%c1,CPU%c3,CPU%c6,CPU%c7,CoreTmp,PkgTmp,GFX%rc6,GFXMHz,PkgWatt,CorWatt,GFXWatt
Signed-off-by: NLen Brown <len.brown@intel.com>

The IRQ column has been working for periodic mode,
but not in one-shot command mode, it shows only 0.

until now.
Signed-off-by: NLen Brown <len.brown@intel.com>

With the --cpu parameter, turbostat prints only lines
for the specified set of CPUs:

sudo ./turbostat --quiet --show Core,CPU --cpu 0,1,3..5,6-7
	Core	CPU
	-	-
	0	0
	0	4
	1	1
	1	5
	2	6
	3	3
	3	7
Signed-off-by: NLen Brown <len.brown@intel.com>

When turbostat shows % of time in a CPU idle power state,
it has always been showing information from underlying
hardware residency counters.

While this reflects what the hardware is doing, and is thus
useful for understanding the hardware,
it doesn't directly tell us what Linux requested --
which is useful for tuning Linux itself.

Here we add columns to turbostat to show the
Linux cpuidle sub-system statistics:
/sys/devices/system/cpu/cpu*/cpuidle/state*/*

The first group of columns are the "usage", which is the
number of times software requested that C-state in the
measurement interval. eg C1 below.

The second group of columns are the "time", which is the percentage
of the measurement interval time that software has requested
the specified C-state. eg C1% below.

These software counters can be compared to the underlying
hardware residency counters (eg CPU%c1	CPU%c3	CPU%c6	CPU%c7)
to compare what sofware requested to what the hardware delivered.

These sysfs attributes are discovered when turbostat starts,
rather than being "built in".  So the --show and --hide
parameters do not know about these dynamic column names.
However "--show sysfs" and "--hide sysfs" act on the
entire group of columns:

turbostat --show sysfs
...
cpu4: POLL: CPUIDLE CORE POLL IDLE
cpu4: C1: MWAIT 0x00
cpu4: C1E: MWAIT 0x01
cpu4: C3: MWAIT 0x10
cpu4: C6: MWAIT 0x20
cpu4: C7s: MWAIT 0x32
...
C1 	C1E	C3 	C6 	C7s	C1% 	C1E%	C3%	C6% 	C7s%
3	6	5	1	188	0.00	0.02	0.00	0.00	99.93
0	6	5	0	58	0.00	0.16	0.02	0.00	99.70
0	0	0	0	9	0.00	0.00	0.00	0.00	99.96
0	0	0	1	24	0.00	0.00	0.00	0.02	99.93
0	0	0	0	9	0.00	0.00	0.00	0.00	99.97
0	0	0	0	32	0.00	0.00	0.00	0.00	99.96
0	0	0	0	7	0.00	0.00	0.00	0.00	99.98
2	0	0	0	36	0.00	0.00	0.00	0.00	99.97
1	0	0	0	13	0.00	0.00	0.00	0.00	99.98
Signed-off-by: NLen Brown <len.brown@intel.com>

Previously, the --add option could specify only an MSR.

Here is is extended so an arbitrary /sys attribute,
as specified by an absolute file path name.

sudo ./turbostat --add /sys/devices/system/cpu/cpu0/cpuidle/state5/usage
Signed-off-by: NLen Brown <len.brown@intel.com>

Skip these two counters on BDX, as they are always zero:
cc7, pc7
Signed-off-by: NLen Brown <len.brown@intel.com>

Newer processors do not hard-code the the number of cpus in each bin
to {1, 2, 3, 4, 5, 6, 7, 8}  Rather, they can specify any number
of CPUS in each of the 8 bins:

eg.

...
37 * 100.0 = 3600.0 MHz max turbo 4 active cores
38 * 100.0 = 3700.0 MHz max turbo 3 active cores
39 * 100.0 = 3800.0 MHz max turbo 2 active cores
39 * 100.0 = 3900.0 MHz max turbo 1 active cores

could now look something like this:

...
37 * 100.0 = 3600.0 MHz max turbo 16 active cores
38 * 100.0 = 3700.0 MHz max turbo 8 active cores
39 * 100.0 = 3800.0 MHz max turbo 4 active cores
39 * 100.0 = 3900.0 MHz max turbo 2 active cores
Signed-off-by: NLen Brown <len.brown@intel.com>

Skip these four counters on SKX, as they are always zero:
cc3, pc3
cc7, pc7
Signed-off-by: NLen Brown <len.brown@intel.com>

The CC1 column in tubostat can be computed by subtracting
the core c-state residency countes from the total Cx residency.

CC1 = (Idle_time_as_measured by MPERF) - (all core C-states with
residency counters)

However, as the underlying counter reads are not atomic,
error can be noticed in this calculations, especially
when the numbers are small.

Denverton has a hardware CC1 residency counter
to improve the accuracy of the cc1 statistic -- use it.

At the same time, Denverton has no concept of CC3, PC3, CC7, PC7,
so skip collecting and printing those columns.

Finally, a note of clarification.
Turbostat prints the standard PC2 residency counter,
but on Denverton hardware, that actually means PC1E.
Turbostat prints the standard PC6 residency counter,
but on Denverton hardware, that actually means PC2.

At this point, we document that differnce in this commit message,
rather than adding a quirk to the software.
Signed-off-by: NLen Brown <len.brown@intel.com>

Gemini Lake is similar to Apollo Lake (Broxton/Goldmont)
Signed-off-by: NLen Brown <len.brown@intel.com>

Fix a bug with --add, where the title of the column
is un-initialized if not specified by the user.

The initial implementation of --show and --hide
neglected to handle the pc8/pc9/pc10 counters.

Fix a bug where "--show Core" only worked with --debug
Reported-by: NWendy Wang <wendy.wang@intel.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

The CPU ticks at a rate in the "bus clock" domain.
eg. 100 MHz * bus_ratio.

On newer processors, the TSC has been moved out of this BCLK
domain and into a separate crystal-clock domain.

While the TSC ticks "close to" the base frequency, those that look
closely at the numbers will notice small errors in calculations that
mix units of TSC clocks and bus clocks.

"tsc_tweak" was introduced to address the most visible
mixing -- the %Busy and the the Busy_MHz calculations.
(A simplification as since removed TSC from the BusyMHz calculation)

Here we apply the tsc_tweak to everyplace where BCLK
and TSC units are mixed.  The results is that
on a system which is 100% idle, the sum of the C-states
are now much more likely to be closer to 100%.
Reported-by: NTravis Downs <travis.downs@gmail.com>
Signed-off-by: NLen Brown <len.brown@intel.com>

Some users want turbostat to tell them everything, by default.
Some users want turbostat to be quiet, by default.

I find that I'm in the 1st camp, and so I've never liked
needing to type the --debug parameter to decode the system
configuration.

So here we change the default and print the system configuration,
by default.  (The --debug option is now un-documented, though
it does still exist for debugging turbostat internals)

When you do not want to see the system configuration
header, use the new "--quiet" option.
Signed-off-by: NLen Brown <len.brown@intel.com>

Some time ago, turbostat overflowed 80 columns.

So on the assumption that a "casual" user would always
want topology and frequency columns, we hid the rest
of the columns and the system configuration decoding
behind the --debug option.

Not everybody liked that change -- including me.
I use --debug 99% of the time...

Well, now we have "-o file" to put turbostat output into a file,
so unless you are watching real-time in a small window,
column count is less frequently a factor.

And more recently, we got the "--hide columnA,columnB" option
to specify columns to skip.

So now we "un-hide" the rest of the columns from behind --debug,
and show them all, by default.
Signed-off-by: NLen Brown <len.brown@intel.com>

useful for observing if the BIOS disabled prefetch
Not architectural, but docuemented as present on NHM, SNB
and is present on others.
Signed-off-by: NLen Brown <len.brown@intel.com>

show the CPUID feature for turbo to clarify the case
when it may not be shown in MISC_ENABLE

CPUID(6): APERF, TURBO, DTS, PTM, No-HWP, No-HWPnotify, No-HWPwindow, No-HWPepp, No-HWPpkg, EPB
cpu4: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT TURBO)
Signed-off-by: NLen Brown <len.brown@intel.com>

Turbostat dumps MSR_TURBO_RATIO_LIMIT on Core Architecture.
But Atom Architecture uses MSR_ATOM_CORE_RATIOS and
MSR_ATOM_CORE_TURBO_RATIOS.
Signed-off-by: NLen Brown <len.brown@intel.com>

Decode MISC_ENABLE.NO_TURBO,
also use the #defines in msr-index.h for decoding this register

cpu0: MSR_IA32_MISC_ENABLE: 0x00850089 (TCC EIST MWAIT TURBO)

Although it is not architectural, decode also
MSR_IA32_MISC_ENABLE.prefetch-disable (bit-9).
documented to be present on: Core, P4, Intel-Xeon
reserved on: Atom, Silvermont, Nehalem, SNB, PHI ec.
Signed-off-by: NLen Brown <len.brown@intel.com>