提交 b91fd9b6 编写于 作者: K Kweku Adams 提交者: Kweku Adams

Updating Battery Historian.

* Updated parsing to work on bug reports from Android Marshmallow and Android N developer preview builds.
* Refreshed and improved UI.
* Ability to view kernel trace data, graph powermonitor readings, and compare bug reports.
* Including some ActivityManager and logcat information, including Bluetooth scans, in the timeline.
* Added support for uploading .zip files directly.
* Improved setup flow and instructions.
* Lots of bug fixes.

Change-Id: I1af14b2598ec6f5466a03744ada14bf79f35431d
上级 fa8b40fd
Copyright 2016 Google Inc. All rights reserved.
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Battery Historian 2.0
=====================
# Battery Historian
Battery Historian is a tool to inspect battery related information and events on an Android device (Android 5.0 Lollipop and later: API Level 21+) while the device was on battery. It allows application developers to visualize system and application level events on a timeline and easily see various aggregated statistics since the device was last fully charged.
Battery Historian is a tool to inspect battery related information and events on an Android device running Android 5.0 Lollipop (API level 21) and later, while the device was on battery. It allows application developers to visualize system and application level events on a timeline with panning and zooming functionality, easily see various aggregated statistics since the device was last fully charged, and select an application and inspect the metrics that impact battery specific to the chosen application. It also allows an A/B comparison of two bugreports, highlighting differences in key battery related metrics.
Introduction
------------
Battery Historian 2.0 is a complete rewrite in Go and uses some JavaScript visualization libraries to display battery related events on a timeline with panning and zooming functionality. In addition, v2.0 allows developers to pick an application and inspect the metrics that impact battery specific to the chosen application.
## Getting Started
Getting Started
---------------
If you are new to the Go programming language:
* Follow the instructions available at <http://golang.org/doc/install> for downloading and installing the Go compilers, tools, and libraries.
* Create a workspace directory according to the instructions at
<http://golang.org/doc/code.html#Organization> and ensure that `GOPATH` and
`GOBIN` environment variables are appropriately set and added to your `$PATH`
<http://golang.org/doc/code.html#Organization>.
* Ensure that `GOPATH` and `GOBIN` environment variables are appropriately set and added to your `$PATH`
environment variable. `$GOBIN should be set to $GOPATH/bin`.
* For Windows, you may set environment variables through the "Environment Variables" button on the
"Advanced" tab of the "System" control panel. Some versions of Windows provide this control panel
through the "Advanced System Settings" option inside the "System" control panel.
* For Linux and Mac OS X, you can add the following lines to your ~/.bashrc or
~/.profile files (assuming your workspace is $HOME/work):
Next, install Go support for Protocol Buffers by running go get.
```
export GOPATH=$HOME/work
export GOBIN=$GOPATH/bin
export PATH=$PATH:$GOBIN
```
```
# Grab the code from the repository and install the proto package.
$ go get -u github.com/golang/protobuf/proto
$ go get -u github.com/golang/protobuf/protoc-gen-go
```
Next, install Git from <https://git-scm.com/downloads> if it's not already installed.
The compiler plugin, protoc-gen-go, will be installed in $GOBIN, which must be
in your $PATH for the protocol compiler, protoc, to find it.
Next, make sure Python 2.7 (NOT Python 3!) is installed. See <https://python.org/downloads>
if it isn't, and ensure that python is added to your `$PATH` environment variable.
Next, download the Battery Historian 2.0 code:
Next, install Java from <http://www.oracle.com/technetwork/java/javase/downloads/index.html>.
Next, download the Battery Historian code and its dependencies:
```
# Download Battery Historian 2.0
$ go get -u github.com/google/battery-historian/...
$ go get -d github.com/google/battery-historian/...
```
Finally, run Battery Historian!
```
$ cd $GOPATH/src/github.com/google/battery-historian
# Compile Javascript files using the Closure compiler
$ bash setup.sh
$ go run setup.go
# Run Historian on your machine (make sure $PATH contains $GOBIN)
$ go run cmd/battery-historian/battery-historian.go [--port <default:9999>]
......@@ -57,19 +60,41 @@ go run cmd/battery-historian/battery-historian.go [--port <default:9999>]
To take a bug report from your Android device, you will need to enable USB debugging under `Settings > System > Developer Options`. On Android 4.2 and higher, the Developer options screen is hidden by default. You can enable this by following the instructions [here](<http://developer.android.com/tools/help/adb.html#Enabling>).
Next, to obtain a bug report from your development device
To obtain a bug report from your development device:
```
$ adb bugreport > bugreport.txt
```
### Start analyzing!
You are all set now. Run `historian` and visit <http://localhost:9999> and upload the `bugreport.txt` file to start analyzing.
## Screenshots
![Timeline](/screenshots/timeline.png "Timeline Visualization")
![System](/screenshots/system.png "Aggregated System statistics since the device was last fully charged")
![History](/screenshots/history.png "Aggregated statistics during distinct discharge sessions")
![App](/screenshots/app.png "Application specific statistics")
## Advanced
To reset aggregated battery stats and history:
```
adb shell dumpsys batterystats --reset
```
##### Wakelock analysis
By default, Android does not record timestamps for application-specific
userspace wakelock transitions even though aggregate statistics are maintained
on a running basis. If you want Historian to display detailed information about
each individual wakelock on the timeline, you should enable full wakelock reporting using the following command before starting your experiment:
each individual wakelock on the timeline, you should enable full wakelock
reporting using the following command before starting your experiment:
```
adb shell dumpsys batterystats --enable full-wake-history
......@@ -78,47 +103,126 @@ adb shell dumpsys batterystats --enable full-wake-history
Note that by enabling full wakelock reporting the battery history log overflows
in a few hours. Use this option for short test runs (3-4 hrs).
To reset aggregated battery stats and timeline at the beginning of a measurement:
##### Kernel trace analysis
To generate a trace file which logs kernel wakeup source and kernel wakelock
activities:
First, connect the device to the desktop/laptop and enable kernel trace logging:
```
$ adb root
$ adb shell
# Set the events to trace.
$ echo "power:wakeup_source_activate" >> /d/tracing/set_event
$ echo "power:wakeup_source_deactivate" >> /d/tracing/set_event
# The default trace size for most devices is 1MB, which is relatively low and might cause the logs to overflow.
# 8MB to 10MB should be a decent size for 5-6 hours of logging.
$ echo 8192 > /d/tracing/buffer_size_kb
$ echo 1 > /d/tracing/tracing_on
```
Then, use the device for intended test case.
Finally, extract the logs:
```
$ echo 0 > /d/tracing/tracing_on
$ adb pull /d/tracing/trace <some path>
# Take a bug report at this time.
$ adb bugreport > bugreport.txt
```
Note:
Historian plots and relates events in real time (PST or UTC), whereas kernel
trace files logs events in jiffies (seconds since boot time). In order to relate
these events there is a script which approximates the jiffies to utc time. The
script reads the UTC times logged in the dmesg when the system suspends and
resumes. The scope of the script is limited to the amount of time stamps present
in the dmesg. Since the script uses the dmesg log when the system suspends,
there are different scripts for each of the device, with only difference being
the device specific dmesg log it tries to find. These scripts have been
integrated into the Battery Historian tool itself.
##### Powermonitor analysis
Powermonitor files should have the following format per line:
```
<timestamp in epoch seconds> <amps>
```
Entries from the powermonitor file will be overlaid on top of the timeline plot.
To ensure the powermonitor and bug report timelines are somewhat aligned,
please reset the batterystats before running any powermonitor logging:
```
adb shell dumpsys batterystats --reset
```
Screenshots
-----------
![Visualization](/screenshots/viz.png "Timeline Visualization")
And take a bug report soon after stopping powermonitor logging.
![System](/screenshots/stats.png "Aggregated System statistics since the device was last fully charged")
If using a Monsoon:
![App](/screenshots/app.png "Application specific statistics")
Download the AOSP Monsoon Python script from <https://android.googlesource.com/platform/cts/+/master/tools/utils/monsoon.py>
```
# Run the script.
$ monsoon.py --serialno 2294 --hz 1 --samples 100000 -timestamp | tee monsoon.out
Advanced
--------
The following information is for advanced users only who are interested in modifying the code.
# ...let device run a while...
$ stop monsoon.py
```
##### Modifying the proto files
If you modify the proto files (pb/\*/\*.proto), you will need to regenerate the compiled Go output files using `regen_proto.sh`.
If you want to modify the proto files (pb/\*/\*.proto), first download the additional tools necessary:
Install the standard C++ implementation of protocol buffers from <https://github.com/google/protobuf/blob/master/src/README.md>
Download the Go proto compiler:
```
$ go get -u github.com/golang/protobuf/protoc-gen-go
```
The compiler plugin, protoc-gen-go, will be installed in $GOBIN, which must be
in your $PATH for the protocol compiler, protoc, to find it.
Make your changes to the proto files.
Finally, regenerate the compiled Go proto output files using `regen_proto.sh`.
##### Other command line tools
```
# System stats
$ go run exec/local_checkin_parse.go --input=bugreport.txt
$ go run cmd/checkin-parse/local_checkin_parse.go --input=bugreport.txt
# Timeline analysis
$ go run exec/local_history_parse.go --summary=totalTime --input=bugreport.txt
$ go run cmd/history-parse/local_history_parse.go --summary=totalTime --input=bugreport.txt
# Diff two bug reports
$ go run cmd/checkin-delta/local_checkin_delta.go --input=bugreport_1.txt,bugreport_2.txt
```
Support
-------
## Support
- G+ Community (Discussion Thread: Battery Historian): https://plus.google.com/b/108967384991768947849/communities/114791428968349268860
If you've found an error in this sample, please file an issue:
<https://github.com/google/battery-historian/issues>
License
-------
## License
Copyright 2016 Google, Inc.
......
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package activity parses activity manager events in bugreport files and outputs CSV entries for those events.
package activity
import (
"bytes"
"fmt"
"regexp"
"sort"
"strconv"
"strings"
"time"
"github.com/google/battery-historian/bugreportutils"
"github.com/google/battery-historian/csv"
"github.com/google/battery-historian/historianutils"
"github.com/google/battery-historian/packageutils"
usagepb "github.com/google/battery-historian/pb/usagestats_proto"
)
var (
// logEntryRE is a regular expression that matches the common prefix to event log and logcat lines in the bug report.
// The details are then matched with the various log event types below.
// e.g. 11-19 11:29:07.341 2206 2933 I
logEntryRE = regexp.MustCompile(`^(?P<month>\d+)-(?P<day>\d+)` + `\s+` + `(?P<timeStamp>[^.]+)` + `[.]` + `(?P<remainder>\d+)` + `\s+` + `(?P<pid>\d+)` + `\s+\d+\s+\S+\s+` + `(?P<details>.*)`)
// activityManagerRE is a regular expression that matches activity manager events.
activityManagerRE = regexp.MustCompile(`^(?P<transitionType>am_(proc_start|proc_died|low_memory|anr))\s*:` + `\s+` + `\[?(?P<value>[^\]]+)\]?`)
// bluetoothScanRE is a regular expression that matches bluetooth scan events.
bluetoothScanRE = regexp.MustCompile(`^.*BluetoothAdapter: startLeScan()`)
// crashStartRE is a regular expression that matches the first line of a crash event.
crashStartRE = regexp.MustCompile(`^AndroidRuntime:\s+` + `FATAL\sEXCEPTION:\s+` + `(?P<source>.+)`)
// crashProcessRE is a regular expression that matches the process information of a crash event.
crashProcessRE = regexp.MustCompile(`^AndroidRuntime:\s+` + `Process:\s(?P<process>\S+)` + `\s*,\s*` + `PID:\s(?P<pid>.+)`)
)
const (
// ProcStartEvent is the string for matching application process start events in the bug report.
ProcStartEvent = "am_proc_start"
// ProcDiedEvent is the string for matching application process died events in the bug report.
ProcDiedEvent = "am_proc_died"
// ANREvent is the string for matching application not responding events in the bug report.
ANREvent = "am_anr"
// LowMemoryEvent is the string for matching low memory events in the bug report.
LowMemoryEvent = "am_low_memory"
// AMProc is the CSV description for the Activity Manager Process related events.
AMProc = "Activity Manager Proc"
)
// procEntry stores the timestamp and details extracted from an am_proc_start or am_proc_died event.
type procEntry struct {
start int64
pid string
uid string
process string
component string
}
// Methods required by csv.EntryState.
func (e *procEntry) GetStartTime() int64 {
return e.start
}
func (e *procEntry) GetType() string {
return "service"
}
func (e *procEntry) GetValue() string {
return fmt.Sprintf("%v~%v~%v~%v", e.pid, e.uid, e.process, e.component)
}
func (e *procEntry) GetKey(desc string) csv.Key {
return csv.Key{
desc,
// The PID is unique while the process is still running.
e.pid,
}
}
type parser struct {
// referenceYear is the year extracted from the dumpstate line in a bugreport. Event log lines don't contain a year in the date string, so we use this to reconstruct the full timestamp.
referenceYear int
// referenceMonth is the month extracted from the dumpstate line in a bugreport. Since a bugreport may span over a year boundary, we use the month to check whether the year for the event needs to be decremented or incremented.
referenceMonth time.Month
// loc is the location parsed from timezone information in the bugreport. The event log is in the user's local timezone which we need to convert to UTC time.
loc *time.Location
// activeProcMap holds the currently active am_proc_start events.
activeProcMap map[string]*procEntry
// buf is the buffer to write the CSV events to.
buf *bytes.Buffer
// csvState stores and prints out events in CSV format.
csvState *csv.State
// pidMappings maps from PID to app info.
pidMappings map[string][]bugreportutils.AppInfo
}
// newParser creates a parser for the given bugreport.
func newParser(br string) (*parser, []string, error) {
loc, err := bugreportutils.TimeZone(br)
if err != nil {
return nil, []string{}, err
}
pm, warnings := bugreportutils.ExtractPIDMappings(br)
// Extract the year and month from the bugreport dumpstate line.
d, err := bugreportutils.DumpState(br)
if err != nil {
return nil, warnings, fmt.Errorf("could not find dumpstate information in the bugreport: %v", err)
}
buf := new(bytes.Buffer)
return &parser{
referenceYear: d.Year(),
referenceMonth: d.Month(),
loc: loc,
activeProcMap: make(map[string]*procEntry),
buf: buf,
csvState: csv.NewState(buf, false),
pidMappings: pm,
}, warnings, nil
}
// fullTimestamp constructs the unix ms timestamp from the given date and time information.
// Since event log events have no corresponding year, we reconstruct the full timestamp using
// the stored reference year and month extracted from the dumpstate line of the bug report.
func (p *parser) fullTimestamp(month, day, partialTimestamp, remainder string) (int64, error) {
parsedMonth, err := strconv.Atoi(month)
if err != nil {
return 0, err
}
if !validMonth(parsedMonth) {
return 0, fmt.Errorf("invalid month: %d", parsedMonth)
}
year := p.referenceYear
// The reference month and year represents the time the bugreport was taken.
// If the bug report event log begins near the end of a year, and rolls over to the next year,
// the events will be in either the previous year to the reference year or in the reference year.
// Bug reports are assumed to span at most a month, but we leave a slightly larger margin here
// in case we get a slightly longer bug report.
if p.referenceMonth < time.March && time.Month(parsedMonth) > time.October {
year--
// Some events may still occur after the given reference date, so we check for a year rollover.
} else if p.referenceMonth > time.October && time.Month(parsedMonth) < time.March {
year++
}
return bugreportutils.TimeStampToMs(fmt.Sprintf("%d-%s-%s %s", year, month, day, partialTimestamp), remainder, p.loc)
}
// Parse writes a CSV entry for each line matching activity manager proc start and died, ANR and low memory events.
// Package info is used to match crash events to UIDs. Errors encountered during parsing will be collected into an errors slice and will continue parsing remaining events.
func Parse(pkgs []*usagepb.PackageInfo, f string) (string, []string, []error) {
p, warnings, err := newParser(f)
if err != nil {
return "", nil, []error{err}
}
var errs []error
crashSource := ""
for _, line := range strings.Split(f, "\n") {
m, result := historianutils.SubexpNames(logEntryRE, line)
if !m {
continue
}
timestamp, err := p.fullTimestamp(result["month"], result["day"], result["timeStamp"], result["remainder"])
if err != nil {
errs = append(errs, err)
continue
}
details := result["details"]
pid := result["pid"]
if m, _ = historianutils.SubexpNames(bluetoothScanRE, details); m {
p.parseBluetoothScan(timestamp, pid)
continue
}
if m, result = historianutils.SubexpNames(crashStartRE, details); m {
crashSource = result["source"]
continue
}
if m, result = historianutils.SubexpNames(crashProcessRE, details); m && crashSource != "" {
var uid string
pkg, err := packageutils.GuessPackage(result["process"], "", pkgs)
if err != nil {
errs = append(errs, err)
// Still want to show the crash event even if there was an error matching a package.
} else if pkg != nil {
uid = fmt.Sprintf("%d", pkg.GetUid())
}
p.csvState.PrintInstantEvent(csv.Entry{
Desc: "Crashes",
Start: timestamp,
Type: "service",
Value: fmt.Sprintf("%s: %s", result["process"], crashSource),
Opt: uid,
})
crashSource = ""
continue
}
m, result = historianutils.SubexpNames(activityManagerRE, details)
if !m {
// Non matching lines are ignored but not considered errors.
continue
}
t := result["transitionType"]
// Format of the value is defined at frameworks/base/services/core/java/com/android/server/am/EventLogTags.logtags.
v := result["value"]
switch t {
case LowMemoryEvent:
p.parseLowMemory(timestamp, v)
case ANREvent:
warning, err := p.parseANR(pkgs, timestamp, v)
if err != nil {
errs = append(errs, err)
}
if warning != "" {
warnings = append(warnings, warning)
}
case ProcStartEvent, ProcDiedEvent:
warning, err := p.parseProc(timestamp, v, t)
if err != nil {
errs = append(errs, err)
}
if warning != "" {
warnings = append(warnings, warning)
}
default:
errs = append(errs, fmt.Errorf("unknown transition for %q: %q", AMProc, t))
}
}
// If there was no corresponding am_proc_died event, set the end time to 0.
p.csvState.PrintAllReset(0)
return p.buf.String(), warnings, errs
}
func (p *parser) parseBluetoothScan(timestamp int64, pid string) {
var appName string
var uid string
apps, ok := p.pidMappings[pid]
if !ok {
appName = fmt.Sprintf("Unknown PID %s", pid)
} else {
// Append the names together in case there's more than one app info.
var names []string
for _, app := range apps {
names = append(names, app.Name)
}
sort.Strings(names)
appName = strings.Join(names, "|")
// Only use the UID info if there's one mapping.
if len(apps) == 1 {
// TODO: consider sharedUserID info.
uid = apps[0].UID
}
}
p.csvState.PrintInstantEvent(csv.Entry{
Desc: "Bluetooth Scan",
Start: timestamp,
Type: "service",
Value: fmt.Sprintf("%s (PID: %s)", appName, pid),
Opt: uid,
})
}
func (p *parser) parseLowMemory(timestamp int64, v string) {
// The value is the number of processes.
p.csvState.PrintInstantEvent(csv.Entry{
Desc: "AM Low Memory",
Start: timestamp,
Type: "service",
Value: v,
})
}
func (p *parser) parseANR(pkgs []*usagepb.PackageInfo, timestamp int64, v string) (string, error) {
// Expected format of v is: User,pid,Package Name,Flags,reason.
parts := strings.Split(v, ",")
if len(parts) < 5 {
return "", fmt.Errorf("%s: got %d parts, want 5", ANREvent, len(parts))
}
warning := ""
if len(parts) > 5 {
warning = fmt.Sprintf("%s: got %d parts, expected 5", ANREvent, len(parts))
}
var uid string
// ANR event should still be displayed even if uid could not be matched.
// Any error is returned at end of function.
pkg, err := packageutils.GuessPackage(parts[2], "", pkgs)
if pkg != nil {
uid = fmt.Sprintf("%d", pkg.GetUid())
}
// We store the UID as part of the ANR value rather than in the Opt field.
// Usually the Opt field is used to populate a service mapper in the JS, however a less roundabout way is to just have the UID as part of the event itself, which will be specially parsed in the JS code.
parts = append(parts[1:5], uid)
p.csvState.PrintInstantEvent(csv.Entry{
Desc: "ANR",
Start: timestamp,
Type: "service",
Value: strings.Join(parts, "~"),
})
return warning, err
}
func (p *parser) parseProc(timestamp int64, v string, t string) (string, error) {
e, warning, err := procEvent(timestamp, v, t)
if err != nil {
return warning, err
}
storedEvent, alreadyActive := p.activeProcMap[e.pid]
switch t {
case ProcStartEvent:
if alreadyActive {
// Double positive transition. Ignore the event.
return warning, fmt.Errorf("two positive transitions for %q, value %q", AMProc, v)
}
// Store the new event.
p.activeProcMap[e.pid] = e
p.csvState.AddEntryWithOpt(AMProc, e, timestamp, e.uid)
return warning, nil
case ProcDiedEvent:
if !alreadyActive {
// No corresponding start event.
p.csvState.AddEntryWithOpt(AMProc, e, 0, e.uid)
p.csvState.AddEntryWithOpt(AMProc, e, timestamp, e.uid)
return warning, nil
}
// Corresponding start event exists, complete the event with the current timestamp.
p.csvState.AddEntryWithOpt(AMProc, storedEvent, timestamp, storedEvent.uid)
delete(p.activeProcMap, storedEvent.pid)
return warning, nil
default:
return warning, fmt.Errorf("unknown transition: %v", t)
}
}
// procEvent returns a procEntry event from the am_proc_start of am_proc_died event.
// If extra fields are encountered, a warning is returned. If fields are missing, an error is returned.
func procEvent(start int64, v string, t string) (*procEntry, string, error) {
warning := ""
switch t {
case ProcStartEvent:
// Expected format of v is: User,PID,UID,Process Name,Type,Component.
parts := strings.Split(v, ",")
if len(parts) < 6 {
return nil, warning, fmt.Errorf("%s: got %d parts, want 6", ProcStartEvent, len(parts))
}
if len(parts) > 6 {
warning = fmt.Sprintf("%s: got %d parts, expected 6", ProcStartEvent, len(parts))
}
if _, err := strconv.Atoi(parts[1]); err != nil {
return nil, warning, fmt.Errorf("%s: could not parse pid %v: %v", ProcStartEvent, parts[1], err)
}
uid, err := packageutils.AppIDFromString(parts[2])
if err != nil {
return nil, warning, fmt.Errorf("%s: could not parse uid %v: %v", ProcStartEvent, parts[2], err)
}
return &procEntry{
start: start,
pid: parts[1],
uid: fmt.Sprint(uid),
process: parts[3],
component: parts[5],
}, warning, nil
case ProcDiedEvent:
// Expected format of v is: User,PID,Process Name.
parts := strings.Split(v, ",")
if len(parts) < 3 {
return nil, warning, fmt.Errorf("%s: got %d parts, want 3", ProcDiedEvent, len(parts))
}
if len(parts) > 3 {
warning = fmt.Sprintf("%s: got %d parts, expected 3", ProcDiedEvent, len(parts))
}
if _, err := strconv.Atoi(parts[1]); err != nil {
return nil, warning, fmt.Errorf("%s: could not parse pid %v: %v", ProcDiedEvent, parts[1], err)
}
return &procEntry{
start: start,
pid: parts[1],
process: parts[2],
}, warning, nil
default:
return nil, "", fmt.Errorf("unknown transition: %v", t)
}
}
func validMonth(m int) bool {
return m >= int(time.January) && m <= int(time.December)
}
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package activity
import (
"errors"
"reflect"
"strings"
"testing"
"github.com/golang/protobuf/proto"
usagepb "github.com/google/battery-historian/pb/usagestats_proto"
)
// TestParse tests the generation of CSV entries for activity manager events from the bug report event logs.
func TestParse(t *testing.T) {
tests := []struct {
desc string
input []string
pkgs []*usagepb.PackageInfo
wantCSV []string
wantWarnings []string
wantErrors []error
}{
{
desc: "Multple am_proc_start and am_proc_died events",
input: []string{
`========================================================`,
`== dumpstate: 2015-09-15 09:51:29`,
`========================================================`,
``,
`09-15 09:29:25.370 29393 31443 I am_proc_start: [11,26187,1110007,com.google.android.gms.unstable,service,com.google.android.gms/.droidguard.DroidGuardService]`,
`09-15 09:29:35.654 29393 30001 I am_proc_start: [11,26297,1110003,android.process.acore,broadcast,com.android.providers.contacts/.PackageIntentReceiver]`,
`09-15 09:32:09.049 29393 30001 I am_proc_died: [11,26187,com.google.android.gms.unstable]`,
`09-15 09:32:11.261 29393 31350 I am_proc_died: [11,26297,android.process.acore]`,
``,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`Activity Manager Proc,service,1442334565370,1442334729049,26187~10007~com.google.android.gms.unstable~com.google.android.gms/.droidguard.DroidGuardService,10007`,
`Activity Manager Proc,service,1442334575654,1442334731261,26297~10003~android.process.acore~com.android.providers.contacts/.PackageIntentReceiver,10003`,
},
},
{
desc: "Different timezone",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-23 13:33:37`,
`========================================================`,
`07-23 12:57:40.883 1917 7187 I am_proc_start: [10,18230,1010068,com.google.android.apps.plus,broadcast,com.google.android.apps.plus/.service.PackagesMediaMonitor]`,
`07-23 12:57:43.546 1917 7187 I am_proc_died: [10,18230,com.google.android.apps.plus]`,
``,
`[persist.sys.timezone]: [Europe/Dublin]`,
},
wantCSV: []string{
`Activity Manager Proc,service,1437652660883,1437652663546,18230~10068~com.google.android.apps.plus~com.google.android.apps.plus/.service.PackagesMediaMonitor,10068`,
},
},
{
desc: "am_proc_start with no corresponding am_proc_died",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-23 13:33:37`,
`========================================================`,
`07-23 12:57:40.883 1917 7187 I am_proc_start: [10,18230,1010068,com.google.android.apps.plus,broadcast,com.google.android.apps.plus/.service.PackagesMediaMonitor]`,
``,
`[persist.sys.timezone]: [Europe/Dublin]`,
},
wantCSV: []string{
`Activity Manager Proc,service,1437652660883,0,18230~10068~com.google.android.apps.plus~com.google.android.apps.plus/.service.PackagesMediaMonitor,10068`,
},
},
{
desc: "am_proc_died with no corresponding am_proc_start",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-23 13:33:37`,
`========================================================`,
`07-23 12:57:43.546 1917 7187 I am_proc_died: [10,18230,com.google.android.apps.plus]`,
``,
`[persist.sys.timezone]: [Europe/Dublin]`,
},
wantCSV: []string{
`Activity Manager Proc,service,0,1437652663546,18230~~com.google.android.apps.plus~,`,
},
},
{
desc: "am_low_memory event",
input: []string{
`========================================================`,
`== dumpstate: 2015-01-27 13:10:19`,
`========================================================`,
`...`,
`01-27 12:32:33.699 745 923 I am_low_memory: 20`,
`01-27 12:32:33.702 745 1203 I force_gc: Binder`,
`01-27 12:32:59.234 745 1290 I am_low_memory: 22`,
`01-27 12:32:59.238 9074 9074 I force_gc: Binder`,
`01-27 12:33:25.381 745 764 I am_low_memory: 23`,
`01-27 12:33:25.386 745 745 I notification_cancel: [10007,28835,com.google.android.gms,10436,NULL,0,0,64,8,NULL]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`AM Low Memory,service,1422390753699,1422390753699,20,`,
`AM Low Memory,service,1422390779234,1422390779234,22,`,
`AM Low Memory,service,1422390805381,1422390805381,23,`,
},
},
{
desc: "am_anr event, some pkg info",
input: []string{
`========================================================`,
`== dumpstate: 2015-09-27 21:04:31`,
`========================================================`,
`...`,
`09-27 20:44:59.609 808 822 I am_anr : [0,2103,com.google.android.gms,-1194836283,executing service com.google.android.gms/.reminders.service.RemindersIntentService]`,
`09-27 20:47:08.686 808 822 I am_anr : [0,3503,com.google.android.gms,-1194836283,Broadcast of Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.gms/.kids.chimera.SystemEventReceiverProxy (has extras) }]`,
`09-27 20:47:08.686 808 822 I am_anr : [0,3503,com.google.android.apps.photos,-1194836283,Broadcast/stuff]`,
`09-27 20:47:08.704 808 1737 I am_proc_bound: [0,3555,com.google.android.apps.photos]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
pkgs: []*usagepb.PackageInfo{
{PkgName: proto.String("com.google.android.apps.photos"), Uid: proto.Int32(1)},
},
wantCSV: []string{
`ANR,service,1443411899609,1443411899609,2103~com.google.android.gms~-1194836283~executing service com.google.android.gms/.reminders.service.RemindersIntentService~,`,
`ANR,service,1443412028686,1443412028686,3503~com.google.android.gms~-1194836283~Broadcast of Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.gms/.kids.chimera.SystemEventReceiverProxy (has extras) }~,`,
`ANR,service,1443412028686,1443412028686,3503~com.google.android.apps.photos~-1194836283~Broadcast/stuff~1,`,
},
},
{
desc: "Year rolls over, event in previous year",
input: []string{
`========================================================`,
`== dumpstate: 2016-01-01 21:04:31`,
`========================================================`,
`...`,
`12-31 20:44:59.609 808 822 I am_anr : [0,2103,com.google.android.gms,-1194836283,reason]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`ANR,service,1451623499609,1451623499609,2103~com.google.android.gms~-1194836283~reason~,`,
},
},
{
desc: "Year rolls over, event matches new year",
input: []string{
`========================================================`,
`== dumpstate: 2016-01-01 21:04:31`,
`========================================================`,
`...`,
`01-01 20:44:59.609 808 822 I am_anr : [0,2103,com.google.android.gms,-1194836283,reason]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`ANR,service,1451709899609,1451709899609,2103~com.google.android.gms~-1194836283~reason~,`,
},
},
{
desc: "Last event after dumpstate time",
input: []string{
`========================================================`,
`== dumpstate: 2015-10-20 09:34:16`,
`========================================================`,
`...`,
`10-20 09:35:23.423 4649 6636 I am_low_memory: 37`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`AM Low Memory,service,1445358923423,1445358923423,37,`,
},
},
{
desc: "Event starts and ends in different years",
input: []string{
`========================================================`,
`== dumpstate: 2016-01-01 21:04:31`,
`========================================================`,
`...`,
`12-31 21:29:25.370 29393 31443 I am_proc_start: [11,26187,1110007,com.google.android.gms.unstable,service,com.google.android.gms/.droidguard.DroidGuardService]`,
`01-01 20:44:59.609 29393 30001 I am_proc_died: [11,26187,com.google.android.gms.unstable]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`Activity Manager Proc,service,1451626165370,1451709899609,26187~10007~com.google.android.gms.unstable~com.google.android.gms/.droidguard.DroidGuardService,10007`,
},
},
{
desc: "am_proc_start and am_proc_died warnings and errors",
input: []string{
`========================================================`,
`== dumpstate: 2015-09-15 09:51:29`,
`========================================================`,
``,
`09-15 09:29:25.370 29393 31443 I am_proc_start: [26187,1110007,com.google.android.gms.unstable,service,com.google.android.gms/.droidguard.DroidGuardService]`,
`09-15 09:29:35.654 29393 30001 I am_proc_start: [11,26297,1110003,android.process.acore,broadcast,com.android.providers.contacts/.PackageIntentReceiver,Newfield]`,
`09-15 09:32:09.049 29393 30001 I am_proc_died: [11,com.google.android.gms.unstable]`,
`09-15 09:32:11.261 29393 31350 I am_proc_died: [11,26297,android.process.acore,new]`,
``,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`Activity Manager Proc,service,1442334575654,1442334731261,26297~10003~android.process.acore~com.android.providers.contacts/.PackageIntentReceiver,10003`,
},
wantWarnings: []string{
"am_proc_start: got 7 parts, expected 6",
"am_proc_died: got 4 parts, expected 3",
},
wantErrors: []error{
errors.New("am_proc_start: got 5 parts, want 6"),
errors.New("am_proc_died: got 2 parts, want 3"),
},
},
{
desc: "am_anr warnings and errors",
input: []string{
`========================================================`,
`== dumpstate: 2015-09-27 21:04:31`,
`========================================================`,
`...`,
`09-27 20:44:59.609 808 822 I am_anr : [0,2103,com.google.android.gms,-1194836283,executing service com.google.android.gms/.reminders.service.RemindersIntentService,extrafield]`,
`09-27 20:47:08.686 808 822 I am_anr : [com.google.android.gms,-1194836283,Broadcast of Intent { act=android.net.conn.CONNECTIVITY_CHANGE flg=0x4000010 cmp=com.google.android.gms/.kids.chimera.SystemEventReceiverProxy (has extras) }]`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
wantCSV: []string{
`ANR,service,1443411899609,1443411899609,2103~com.google.android.gms~-1194836283~executing service com.google.android.gms/.reminders.service.RemindersIntentService~,`,
},
wantWarnings: []string{
"am_anr: got 6 parts, expected 5",
},
wantErrors: []error{
errors.New("am_anr: got 3 parts, want 5"),
},
},
{
desc: "Crashes, volta pkg info provided, no pkg info for vending",
input: []string{
`========================================================`,
`== dumpstate: 2015-08-06 15:30:45`,
`========================================================`,
`...`,
`08-05 22:58:11.751 10686 10707 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #1`,
`08-05 22:58:11.751 10686 10707 E AndroidRuntime: Process: com.google.android.volta, PID: 10686`,
`08-05 22:58:11.751 10686 10707 E AndroidRuntime: java.lang.RuntimeException: An error occured while executing doInBackground()`,
`08-05 22:58:11.751 10686 10707 E AndroidRuntime: at android.os.AsyncTask$3.done(AsyncTask.java:304)`,
`08-06 00:35:50.774 23682 23801 E AndroidRuntime: FATAL EXCEPTION: AsyncTask #2`,
`08-06 00:35:50.774 23682 23801 E AndroidRuntime: Process: com.google.android.volta, PID: 23682`,
`08-06 00:35:50.774 23682 23801 E AndroidRuntime: java.lang.RuntimeException: An error occured while executing doInBackground()`,
`08-06 00:35:50.774 23682 23801 E AndroidRuntime: at android.os.AsyncTask$3.done(AsyncTask.java:304)`,
`08-06 00:35:50.774 23682 23801 E AndroidRuntime: at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)`,
`08-06 00:35:50.774 20440 20440 E AndroidRuntime: FATAL EXCEPTION: main`,
`08-06 00:35:50.774 20440 20440 E AndroidRuntime: Process: com.android.vending, PID: 20440`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
},
pkgs: []*usagepb.PackageInfo{
{PkgName: proto.String("com.google.android.volta"), Uid: proto.Int32(1)},
},
wantCSV: []string{
`Crashes,service,1438840691751,1438840691751,com.google.android.volta: AsyncTask #1,1`,
`Crashes,service,1438846550774,1438846550774,com.google.android.volta: AsyncTask #2,1`,
`Crashes,service,1438846550774,1438846550774,com.android.vending: main,`,
},
},
{
desc: "Bluetooth scans, some PID mappings and pkg info",
input: []string{
`========================================================`,
`== dumpstate: 2015-11-05 06:30:29`,
`========================================================`,
`...`,
`11-05 06:19:14.095 1691 5180 D BluetoothAdapter: startLeScan(): null`,
`11-05 06:19:15.815 1691 5180 D BluetoothAdapter: startLeScan(): null`,
`11-05 06:20:10.417 17745 17745 D BluetoothAdapter: startLeScan(): null`,
`...`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
`...`,
` PID mappings:`,
` PID #784: ProcessRecord{b2760e2 784:system/1000}`,
` PID #17745: ProcessRecord{4fe996a 17745:gbis.gbandroid/u0a105}`,
},
wantCSV: []string{
`Bluetooth Scan,service,1446733154095,1446733154095,Unknown PID 1691 (PID: 1691),`,
`Bluetooth Scan,service,1446733155815,1446733155815,Unknown PID 1691 (PID: 1691),`,
`Bluetooth Scan,service,1446733210417,1446733210417,gbis.gbandroid (PID: 17745),10105`,
},
},
}
for _, test := range tests {
output, warnings, errs := Parse(test.pkgs, strings.Join(test.input, "\n"))
got := normalizeCSV(output)
want := normalizeCSV(strings.Join(test.wantCSV, "\n"))
if !reflect.DeepEqual(got, want) {
t.Errorf("%v: Parse(%v)\n outputted csv = %v\n want: %v", test.desc, test.input, strings.Join(got, "\n"), strings.Join(want, "\n"))
}
if !reflect.DeepEqual(errs, test.wantErrors) {
t.Errorf("%v: Parse(%v)\n unexpected errors = %v\n want: %v", test.desc, test.input, errs, test.wantErrors)
}
if !reflect.DeepEqual(warnings, test.wantWarnings) {
t.Errorf("%v: Parse(%v)\n unexpected warnings = %v\n want: %v", test.desc, test.input, warnings, test.wantWarnings)
}
}
}
// Removes trailing space at the end of the string,
// then splits by new line.
func normalizeCSV(text string) []string {
return strings.Split(strings.TrimSpace(text), "\n")
}
此差异已折叠。
此差异已折叠。
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package bugreportutils is a library of common bugreport parsing functions.
package bugreportutils
import (
"archive/zip"
"bytes"
"errors"
"fmt"
"io"
"net/http"
"regexp"
"strconv"
"strings"
"time"
"github.com/google/battery-historian/historianutils"
"github.com/google/battery-historian/packageutils"
)
const (
// GPSSensorNumber is the hard-coded sensor number defined in frameworks/base/core/java/android/os/BatteryStats.Sensor
GPSSensorNumber = -10000
// TimeLayout is the timestamp layout commonly printed in bug reports.
TimeLayout = "2006-01-02 15:04:05"
)
var (
// bugReportSectionRE is a regular expression to match the beginning of a bug report section.
bugReportSectionRE = regexp.MustCompile(`------\s+(?P<section>.*)\s+-----`)
// deviceIDRE is a regular expression that matches the "DeviceID" line
deviceIDRE = regexp.MustCompile("DeviceID: (?P<deviceID>[0-9]+)")
// sdkVersionRE is a regular expression that finds sdk version in the System Properties section of a bug report
sdkVersionRE = regexp.MustCompile(`\[ro.build.version.sdk\]:\s+\[(?P<sdkVersion>\d+)\]`)
// buildFingerprintRE is a regular expression to match any build fingerprint line in the bugreport
buildFingerprintRE = regexp.MustCompile(`Build\s+fingerprint:\s+'(?P<build>\S+)'`)
// modelNameRE is a regular expression that finds the model name line in the System Properties section of a bug report.
modelNameRE = regexp.MustCompile(`\[ro.product.model\]:\s+\[(?P<modelName>.*)\]`)
// pidRE is a regular expression to match PID to app name and UID.
pidRE = regexp.MustCompile(`PID #` + `(?P<pid>\d+)` + `: ProcessRecord[^:]+:` + `(?P<app>[^/]+)` + `/` + `(?P<uid>.*)` + `}`)
// sensorLineRE is a regular expression to match the sensor list line in the sensorservice dump of a bug report.
sensorLineRE = regexp.MustCompile(`(?P<sensorName>[^|]+)` + `\|` + `(?P<sensorManufacturer>[^|]+)` + `\|` +
`(\s*version=(?P<versionNumber>\d+)\s*\|)?` + `\s*(?P<sensorTypeString>[^|]+)` +
`\|` + `\s*(?P<sensorNumber>0x[0-9A-Fa-f]+)`)
// TimeZoneRE is a regular expression to match the timezone string in a bug report.
TimeZoneRE = regexp.MustCompile(`^\[persist.sys.timezone\]:\s+\[` + `(?P<timezone>\S+)\]`)
// DumpstateRE is a regular expression that matches the time information from the dumpstate line at the start of a bug report.
DumpstateRE = regexp.MustCompile(`==\sdumpstate:\s(?P<timestamp>\d+-\d+-\d+\s\d+:\d+:\d+)`)
)
// Contents returns a map of the contents of each file from the given bytes slice, with the key being the file name.
// Supported file formats are text/plain and application/zip.
// For zipped files, each file name will be prepended by the zip file's name.
// An error will be non-nil for processing issues.
func Contents(fname string, b []byte) (map[string][]byte, error) {
contentType := http.DetectContentType(b)
switch {
case strings.Contains(contentType, "text/plain"):
return map[string][]byte{fname: b}, nil
case strings.Contains(contentType, "application/zip"):
return unzipAndExtract(fname, b)
default:
return nil, fmt.Errorf("incorrect file format detected: %q", contentType)
}
}
// IsBugReport tries to determine if the given bytes resembles a bug report.
func IsBugReport(b []byte) bool {
// Check for a few expected lines in all bug reports.
return DumpstateRE.Match(b) && buildFingerprintRE.Match(b) && bugReportSectionRE.Match(b)
}
// unzipAndExtract unzips the given application/zip format file and returns the contents of each file.
// An error will be non-nil for processing issues.
func unzipAndExtract(fname string, b []byte) (map[string][]byte, error) {
r, err := zip.NewReader(bytes.NewReader(b), int64(len(b)))
if err != nil {
return nil, fmt.Errorf("failed to open ZIP file: %v", err)
}
files := make(map[string][]byte)
for _, f := range r.File {
rc, err := f.Open()
if err != nil {
return nil, fmt.Errorf("error reading from ZIP file: %v", err)
}
defer rc.Close()
var zc bytes.Buffer
_, err = io.Copy(&zc, rc)
if err != nil {
return nil, fmt.Errorf("error copying from ZIP file: %v", err)
}
files[fname+"~"+f.Name] = zc.Bytes()
}
return files, nil
}
// MetaInfo contains metadata about the device being analyzed
type MetaInfo struct {
DeviceID string
SdkVersion int
BuildFingerprint string
ModelName string
Sensors map[int32]SensorInfo
}
// SensorInfo contains basic information about a device's sensor.
type SensorInfo struct {
Name, Type string
Version, Number int32
TotalTimeMs int64 // time.Duration in Golang is converted to nanoseconds in JS, so using int64 and naming convention to be clear in$
Count float32
}
// ParseMetaInfo extracts the device ID, build fingerprint and model name from the bug report.
func ParseMetaInfo(input string) (*MetaInfo, error) {
var deviceID, buildFingerprint, modelName string
sdkVersion := -1
for _, line := range strings.Split(input, "\n") {
if match, result := historianutils.SubexpNames(deviceIDRE, line); match {
deviceID = result["deviceID"]
} else if match, result := historianutils.SubexpNames(sdkVersionRE, line); match {
sdk, err := strconv.Atoi(result["sdkVersion"])
if err != nil {
return nil, err
}
sdkVersion = sdk
} else if match, result := historianutils.SubexpNames(buildFingerprintRE, line); match && buildFingerprint == "" {
// Only the first instance of this line in the bug report is guaranteed to be correct.
// All following instances may be wrong, so we ignore them.
buildFingerprint = result["build"]
} else if match, result := historianutils.SubexpNames(modelNameRE, line); match {
modelName = result["modelName"]
}
if deviceID != "" && buildFingerprint != "" && sdkVersion != -1 && modelName != "" {
break
}
}
if sdkVersion == -1 {
return nil, errors.New("unable to find device SDK version")
}
if deviceID == "" {
deviceID = "not available"
}
if modelName == "" {
modelName = "unknown device"
}
sensors, err := extractSensorInfo(input)
return &MetaInfo{
DeviceID: deviceID,
SdkVersion: sdkVersion,
BuildFingerprint: buildFingerprint,
ModelName: modelName,
Sensors: sensors,
}, err
}
// extractSensorInfo extracts device sensor information found in the sensorservice dump of a bug report.
func extractSensorInfo(input string) (map[int32]SensorInfo, error) {
inSSection := false
sensors := make(map[int32]SensorInfo)
Loop:
for _, line := range strings.Split(input, "\n") {
if m, result := historianutils.SubexpNames(historianutils.ServiceDumpRE, line); m {
switch in := result["service"] == "sensorservice"; {
case inSSection && !in: // Just exited the section
break Loop
case in:
inSSection = true
continue Loop
default: // Random section
continue Loop
}
}
if !inSSection {
continue
}
if m, result := historianutils.SubexpNames(sensorLineRE, line); m {
n, err := strconv.ParseInt(result["sensorNumber"], 0, 32)
if err != nil {
return nil, err
}
v := 0
if x := result["versionNumber"]; x != "" {
v, err = strconv.Atoi(x)
if err != nil {
return nil, err
}
}
sensors[int32(n)] = SensorInfo{
Name: result["sensorName"],
Number: int32(n),
Type: result["sensorTypeString"],
Version: int32(v),
}
}
}
sensors[GPSSensorNumber] = SensorInfo{
Name: "GPS",
Number: GPSSensorNumber,
}
return sensors, nil
}
// ExtractBatterystatsCheckin extracts and returns only the lines in
// input that are included in the "CHECKIN BATTERYSTATS" section.
func ExtractBatterystatsCheckin(input string) string {
inBsSection := false
var bsCheckin []string
Loop:
for _, line := range strings.Split(input, "\n") {
line = strings.TrimSpace(line)
if m, result := historianutils.SubexpNames(bugReportSectionRE, line); m {
switch in := strings.Contains(result["section"], "CHECKIN BATTERYSTATS"); {
case inBsSection && !in: // Just exited the section
break Loop
case in:
inBsSection = true
continue Loop
default: // Random section
continue Loop
}
}
if inBsSection {
bsCheckin = append(bsCheckin, line)
}
}
return strings.Join(bsCheckin, "\n")
}
// ExtractBugReport extracts and returns only the first valid bug report data
// in the given contents. The second returned parameter will be the determined
// file name.
func ExtractBugReport(fname string, contents []byte) (string, string, error) {
fs, err := Contents(fname, contents)
if err != nil {
return "", "", err
}
for n, f := range fs {
if IsBugReport(f) {
return string(f), n, nil
}
}
return "", "", fmt.Errorf("%s did not contain a valid bug report", fname)
}
// AppInfo holds the name and UID for an app.
type AppInfo struct {
Name string
UID string
}
// ExtractPIDMappings returns mappings from PID to app names and UIDs extracted from the bug report.
func ExtractPIDMappings(contents string) (map[string][]AppInfo, []string) {
var warnings []string
mapping := make(map[string][]AppInfo)
for _, line := range strings.Split(contents, "\n") {
if m, result := historianutils.SubexpNames(pidRE, line); m {
baseUID, err := packageutils.AppIDFromString(result["uid"])
uidStr := strconv.Itoa(int(baseUID))
if err != nil {
uidStr = ""
warnings = append(warnings, fmt.Sprintf("invalid uid: %s", result["uid"]))
}
mapping[result["pid"]] = append(mapping[result["pid"]], AppInfo{
Name: result["app"],
UID: uidStr,
})
}
}
return mapping, warnings
}
// TimeStampToMs converts a timestamp in the TimeLayout format, combined with the fraction of a second, to a unix ms timestamp based on the location.
func TimeStampToMs(timestamp, remainder string, loc *time.Location) (int64, error) {
if loc == nil {
return 0, errors.New("missing location")
}
t, err := time.ParseInLocation(TimeLayout, timestamp, loc)
if err != nil {
return 0, err
}
// The remainder represents the fraction of a second. e.g. timestamp 2015-05-28 19:50:27.123456 has remainder 123456.
// The remainder will be parsed as ms, so only the leading 3 digits of the remainder are used.
// Make sure the remainder has at least 3 digits, so the slice operation doesn't fail.
remainder = fmt.Sprintf("%s000", remainder)
// Truncate the remainder to 3 decimal points.
remainder = remainder[:3]
parsedInt, err := strconv.ParseInt(remainder, 10, 64)
if err != nil {
return 0, err
}
return ((t.Unix() * 1000) + parsedInt), nil
}
// TimeZone extracts the time zone from a bug report.
func TimeZone(contents string) (*time.Location, error) {
for _, line := range strings.Split(contents, "\n") {
if m, result := historianutils.SubexpNames(TimeZoneRE, line); m {
return time.LoadLocation(result["timezone"])
}
}
return nil, errors.New("missing time zone line in bug report")
}
// DumpState returns the parsed dumpstate information as a time object.
func DumpState(contents string) (time.Time, error) {
loc, err := TimeZone(contents)
if err != nil {
return time.Time{}, err
}
for _, line := range strings.Split(contents, "\n") {
if m, result := historianutils.SubexpNames(DumpstateRE, line); m {
d, err := time.ParseInLocation(TimeLayout, strings.TrimSpace(result["timestamp"]), loc)
if err != nil {
return time.Time{}, err
}
return d, nil
}
}
return time.Time{}, errors.New("could not find dumpstate information in bugreport")
}
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package bugreportutils
import (
"errors"
"reflect"
"strings"
"testing"
"time"
)
// Tests the conversion of times in the format: "2015-05-28 19:50:27.636636" to unix time in ms.
// This is parsed into two separate strings "2015-05-28 19:50:27" and "636636".
func TestTimeStampToMs(t *testing.T) {
tests := []struct {
desc string
timestamp string
remainder string
loc *time.Location
want int64
wantErr error
}{
{
desc: "UTC location",
timestamp: "2015-05-28 19:50:27",
remainder: "636636",
loc: time.UTC,
want: 1432842627636,
},
{
desc: "An hour east of UTC",
timestamp: "2015-05-28 19:50:27",
remainder: "636636",
loc: time.FixedZone("an hour east of UTC", 3600),
want: (1432842627636 - 3600000),
},
{
desc: "Missing location",
timestamp: "2015-05-28 19:50:27",
remainder: "636636",
wantErr: errors.New("missing location"),
},
{
desc: "Missing remainder",
timestamp: "2015-05-28 19:50:27",
loc: time.UTC,
want: 1432842627000,
},
{
desc: "Length of remainder < 3",
timestamp: "2015-05-28 19:50:27.6",
remainder: "6",
loc: time.UTC,
want: 1432842627600,
},
}
for _, test := range tests {
got, err := TimeStampToMs(test.timestamp, test.remainder, test.loc)
if !reflect.DeepEqual(err, test.wantErr) {
t.Errorf("%v: TimeStampToMs(%v, %v)\n got err: %v\n want err: %v", test.desc, test.timestamp, test.remainder, err, test.wantErr)
}
if got != test.want {
t.Errorf("%v: TimeStampToMs(%v, %v)\n got: %v\n want: %v", test.desc, test.timestamp, test.remainder, got, test.want)
}
}
}
// Tests the extracting of the time zone from a bug report.
func TestTimeZone(t *testing.T) {
tests := []struct {
desc string
input []string
want string
wantErr error
}{
{
desc: "Europe/London time zone",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-07 18:07:00`,
`========================================================`,
``,
`Build: LYZ28H`,
`...`,
`[persist.sys.localevar]: []`,
`[persist.sys.media.use-awesome]: [true]`,
`[persist.sys.profiler_ms]: [0]`,
`[persist.sys.timezone]: [Europe/London]`,
},
want: "Europe/London",
},
{
desc: "America/Los_Angeles time zone",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-31 09:20:54`,
`========================================================`,
``,
`Build: shamu-userdebug M MRA16G 2097933 dev-keys`,
`..`,
`[persist.sys.qc.sub.rdump.on]: [0]`,
`[persist.sys.timezone]: [America/Los_Angeles]`,
`[persist.sys.usb.config]: [adb]`,
`[ril.baseband.config.version]: [SHAMU_TMO_CUST]`,
},
want: "America/Los_Angeles",
},
{
desc: "Invalid time zone",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-31 09:20:54`,
`========================================================`,
``,
`Build: shamu-userdebug M MRA16G 2097933 dev-keys`,
`..`,
`[persist.sys.qc.sub.rdump.on]: [0]`,
`[persist.sys.timezone]: [Invalid]`,
`[persist.sys.usb.config]: [adb]`,
`[ril.baseband.config.version]: [SHAMU_TMO_CUST]`,
},
wantErr: errors.New("unknown time zone Invalid"),
},
{
desc: "Missing time zone",
input: []string{
`========================================================`,
`== dumpstate: 2015-07-31 09:20:54`,
`========================================================`,
``,
`Build: shamu-userdebug M MRA16G 2097933 dev-keys`,
`..`,
`[persist.sys.qc.sub.rdump.on]: [0]`,
`[persist.sys.usb.config]: [adb]`,
`[ril.baseband.config.version]: [SHAMU_TMO_CUST]`,
},
wantErr: errors.New("missing time zone line in bug report"),
},
}
for _, test := range tests {
input := strings.Join(test.input, "\n")
got, err := TimeZone(input)
if !reflect.DeepEqual(err, test.wantErr) {
t.Errorf("%v: TimeZone(%v)\n got err: %v\n want err: %v", test.desc, input, err, test.wantErr)
}
if test.wantErr != nil {
continue
}
if got.String() != test.want {
t.Errorf("%v: TimeZone(%v)\n got: %q\n want: %q", test.desc, input, got.String(), test.want)
}
}
}
// Tests the metaInfo parsing results
func TestParseMetaInfo(t *testing.T) {
tests := []struct {
name, input string
want *MetaInfo
}{
{
name: "ParseMetaInfo (all entries)",
input: strings.Join([]string{
`Build: LRX22C`,
`Build fingerprint: 'google/hammerhead/hammerhead:5.0.1/LRX22C/1602158:user/release-keys'`,
`Bootloader: HHZ12d`,
`Radio: msm`,
`Network: T-Mobile`,
`...`,
`[ro.build.id]: [LRX22C]`,
`[ro.build.version.sdk]: [21]`,
` [ro.product.model]: [Nexus 5]`, // space intentionally added to make sure it doesn't affect extraction
`...`,
`Client:`,
` DeviceID: 123456789012345678`,
`DUMP OF SERVICE sensorservice:`,
`Sensor List:`,
// The spaces are actually found in bug reports.
`AK8963 Magnetometer Uncalibrated| AKM | version=1 |android.sensor.magnetic_field_uncalibrated| 0x0000000b | "" | type=14 | continuous | minRate=1.00Hz | maxRate=60.00Hz | FifoMax=1500 events | non-wakeUp | last=<>`,
`AMD | QTI | version=1 | | 0x00000015 | "" | type=33171006 | on-change | maxDelay=0us |minDelay=0us |no batching | non-wakeUp | last=<>`,
// Same name, different types
`APDS-9930/QPDS-T930 Proximity & Light| Avago | version=2 |android.sensor.proximity| 0x00000024 | "" | type=8 | on-change | maxDelay=0us |minDelay=0us |no batching | wakeUp | last=<> `,
`APDS-9930/QPDS-T930 Proximity & Light| Avago | version=2 |android.sensor.light| 0x00000001 | "" | type=5 | on-change | maxDelay=0us |minDelay=0us |FifoMax=240 events | non-wakeUp | last 10 events = < 1) 8.2, 0.0, 0.0,159312324052923 16:15:19 2) 8.7, 0.0, 0.0,159312374406927 16:15:19 3) 9.0, 0.0, 0.0,159312424760931 16:15:19 4) 9.2, 0.0, 0.0,159315748125189 16:15:22 5) 9.7, 0.0, 0.0,159315798479193 16:15:22 6) 10.2, 0.0, 0.0,159315848833197 16:15:22 7) 10.7, 0.0, 0.0,159315899187201 16:15:22 8) 11.0, 0.0, 0.0,159315949541205 16:15:22 9) 11.2, 0.0, 0.0,159316050249212 16:15:22 10) 11.7, 0.0, 0.0,159316100603216 16:15:22 >`,
// L format
`Accelerometer Sensor| HTC Group Ltd.| android.sensor.accelerometer| 0x00000000 | "" | type=1 | continuous | minRate=5.00Hz | maxRate=100.00Hz | FifoMax=1220 events | non-wakeUp | last=< 0.2, -0.0, 9.7, 2340745539166063>`,
`Gyroscope Sensor (WAKE_UP)| HTC Group Ltd.| android.sensor.gyroscope| 0x0000000c | "" | type=4 | continuous | minRate=5.00Hz | maxRate=100.00Hz | FifoMax=1220 events | wakeUp | last=< 0.0, 0.0, 0.0, 0>`,
}, "\n"),
want: &MetaInfo{
DeviceID: `123456789012345678`,
SdkVersion: 21,
ModelName: "Nexus 5",
BuildFingerprint: `google/hammerhead/hammerhead:5.0.1/LRX22C/1602158:user/release-keys`,
Sensors: map[int32]SensorInfo{
-10000: {
Name: `GPS`,
Number: -10000,
},
0: {
Name: `Accelerometer Sensor`,
Number: 0,
Version: 0,
Type: `android.sensor.accelerometer`,
},
1: {
Name: `APDS-9930/QPDS-T930 Proximity & Light`,
Number: 1,
Version: 2,
Type: `android.sensor.light`,
},
11: {
Name: `AK8963 Magnetometer Uncalibrated`,
Number: 11,
Version: 1,
Type: `android.sensor.magnetic_field_uncalibrated`,
},
12: {
Name: `Gyroscope Sensor (WAKE_UP)`,
Number: 12,
Version: 0,
Type: `android.sensor.gyroscope`,
},
21: {
Name: `AMD`,
Number: 21,
Version: 1,
Type: ``,
},
36: {
Name: `APDS-9930/QPDS-T930 Proximity & Light`,
Number: 36,
Version: 2,
Type: `android.sensor.proximity`,
},
},
},
},
{
name: "ParseMetaInfo (without DeviceID)",
input: strings.Join([]string{
`Build: LRX22C`,
`Build fingerprint: 'google/hammerhead/hammerhead:5.0.1/LRX22C/1602158:user/release-keys'`,
`Bootloader: HHZ12d`,
`Radio: msm`,
`Network: T-Mobile`,
`...`,
`[ro.build.id]: [LRX22C]`,
`[ro.build.version.sdk]: [21]`,
`[ro.product.model]: [Nexus 6]`,
`...`,
`Client:`,
}, "\n"),
want: &MetaInfo{
DeviceID: `not available`,
SdkVersion: 21,
ModelName: "Nexus 6",
BuildFingerprint: `google/hammerhead/hammerhead:5.0.1/LRX22C/1602158:user/release-keys`,
Sensors: map[int32]SensorInfo{
-10000: {Name: `GPS`, Number: -10000},
},
},
},
{
name: "ParseMetaInfo with multiple Build fingerprint lines",
input: strings.Join([]string{
// From the top of a bug report.
`Build: shamu-userdebug 6.0 MRA58E 2219288 dev-keys`,
`Build fingerprint: 'google/shamu/shamu:6.0/MRA58E/2219288:userdebug/dev-keys'`,
`Bootloader: moto-apq8084-71.15`,
`Radio: msm`,
`Network: (unknown)`,
// There can be multiple instances of the following 3 lines in bug reports,
// and in some cases, the build fingerprint will not be the correct one.
`----- pid 10754 at 2015-08-17 01:11:07 -----`,
`Cmd line: random.package.name`,
`Build fingerprint: 'google/shamu/shamu:6.0/MRA42/2155602:userdebug/dev-keys'`,
`...`,
`[ro.build.id]: [MRA58E]`,
`[ro.build.version.sdk]: [23]`,
`[ro.product.model]: [Nexus 6]`,
`...`,
`Client:`,
` DeviceID: 123456789012345678`,
}, "\n"),
want: &MetaInfo{
DeviceID: `123456789012345678`,
SdkVersion: 23,
ModelName: "Nexus 6",
BuildFingerprint: `google/shamu/shamu:6.0/MRA58E/2219288:userdebug/dev-keys`,
Sensors: map[int32]SensorInfo{
-10000: {Name: `GPS`, Number: -10000},
},
},
},
}
for _, test := range tests {
meta, err := ParseMetaInfo(test.input)
if err != nil {
t.Errorf("%v: ParseMetaInfo(%v) error: %q", test.name, test.input, err)
}
if !reflect.DeepEqual(meta, test.want) {
t.Errorf("%v--ParseMetaInfo(%v):\n got: %v\n want: %v", test.name, test.input, meta, test.want)
}
}
}
// Tests getting the PID to app mapping.
func TestExtractPIDMappings(t *testing.T) {
tests := []struct {
desc string
input []string
want map[string][]AppInfo
wantWarnings []string
}{
{
desc: "Various PID mappings",
input: []string{
` PID mappings:`,
` PID #659: ProcessRecord{9b4f852 659:com.motorola.targetnotif/u0a124}`,
` PID #1422: ProcessRecord{96225d2 1422:com.google.android.apps.shopping.express/u0a183}`,
` PID #1805: ProcessRecord{e2a1678 1805:com.facebook.katana/u0a157}`,
},
want: map[string][]AppInfo{
"659": {
{
Name: "com.motorola.targetnotif",
UID: "10124",
},
},
"1422": {
{
Name: "com.google.android.apps.shopping.express",
UID: "10183",
},
},
"1805": {
{
Name: "com.facebook.katana",
UID: "10157",
},
},
},
},
{
desc: "Duplicated mapping",
input: []string{
` PID mappings:`,
` PID #659: ProcessRecord{9b4f852 659:com.motorola.targetnotif/u0a124}`,
` PID #659: ProcessRecord{96225d2 659:com.google.android.apps.shopping.express/u0a183}`,
},
want: map[string][]AppInfo{
"659": {
{
Name: "com.motorola.targetnotif",
UID: "10124",
},
{
Name: "com.google.android.apps.shopping.express",
UID: "10183",
},
},
},
},
{
desc: "Warnings",
input: []string{
` PID mappings:`,
` PID #659: ProcessRecord{9b4f852 659:com.motorola.targetnotif/invaliduid}`,
},
want: map[string][]AppInfo{
"659": {
{
Name: "com.motorola.targetnotif",
},
},
},
wantWarnings: []string{"invalid uid: invaliduid"},
},
}
for _, test := range tests {
pm, warns := ExtractPIDMappings(strings.Join(test.input, "\n"))
if !reflect.DeepEqual(pm, test.want) {
t.Errorf("%v--ExtractPIDMappings(%v):\n got: %v\n want: %v", test.desc, test.input, pm, test.want)
}
if !reflect.DeepEqual(warns, test.wantWarnings) {
t.Errorf("%v--ExtractPIDMappings(%v):\n got warnings: %v\n want: %v", test.desc, test.input, warns, test.wantWarnings)
}
}
}
此差异已折叠。
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package checkindelta
import (
"errors"
"math"
"reflect"
"github.com/golang/protobuf/proto"
bspb "github.com/google/battery-historian/pb/batterystats_proto"
)
// roundToTwoDecimal rounds off floats to 2 decimal places.
func roundToTwoDecimal(val float64) float64 {
var roundedVal float64
if val > 0 {
roundedVal = math.Floor((val * 100) + .5)
} else {
roundedVal = math.Ceil((val * 100) - .5)
}
return roundedVal / 100
}
// NormalizeStats takes in a proto and normalizes it by converting
// any absolute value to value/TotalTime.
func NormalizeStats(p *bspb.BatteryStats) (*bspb.BatteryStats, error) {
totalTimeHour := roundToTwoDecimal(float64((p.GetSystem().GetBattery().GetBatteryRealtimeMsec()) / (3600 * 1000)))
if totalTimeHour == 0 {
return nil, errors.New("battery real time cannot be 0")
}
normApp := &bspb.BatteryStats{
ReportVersion: p.ReportVersion,
AggregationType: p.AggregationType,
}
// Normalize the app data
for _, a1 := range p.GetApp() {
a := normalizeApp(a1, totalTimeHour)
normApp.App = append(normApp.App, a)
}
// Normalize system data
s1 := p.GetSystem()
s := &bspb.BatteryStats_System{}
if norm := normalizeMessage(s1.GetBattery(), totalTimeHour); norm != nil {
s.Battery = norm.(*bspb.BatteryStats_System_Battery)
}
if norm := normalizeMessage(s1.GetBatteryDischarge(), totalTimeHour); norm != nil {
s.BatteryDischarge = norm.(*bspb.BatteryStats_System_BatteryDischarge)
}
s.BatteryLevel = s1.GetBatteryLevel()
if norm := normalizeRepeatedMessage(s1.GetBluetoothState(), totalTimeHour); !norm.IsNil() {
s.BluetoothState = norm.Interface().([]*bspb.BatteryStats_System_BluetoothState)
}
if norm := normalizeRepeatedMessage(s1.GetDataConnection(), totalTimeHour); !norm.IsNil() {
s.DataConnection = norm.Interface().([]*bspb.BatteryStats_System_DataConnection)
}
if norm := normalizeMessage(s1.GetGlobalNetwork(), totalTimeHour); norm != nil {
s.GlobalNetwork = norm.(*bspb.BatteryStats_System_GlobalNetwork)
}
if norm := normalizeRepeatedMessage(s1.GetKernelWakelock(), totalTimeHour); !norm.IsNil() {
s.KernelWakelock = norm.Interface().([]*bspb.BatteryStats_System_KernelWakelock)
}
if norm := normalizeMessage(s1.GetMisc(), totalTimeHour); norm != nil {
s.Misc = norm.(*bspb.BatteryStats_System_Misc)
}
if norm := normalizeRepeatedMessage(s1.GetPowerUseItem(), totalTimeHour); !norm.IsNil() {
s.PowerUseItem = norm.Interface().([]*bspb.BatteryStats_System_PowerUseItem)
}
if norm := normalizeMessage(s1.GetPowerUseSummary(), totalTimeHour); norm != nil {
s.PowerUseSummary = norm.(*bspb.BatteryStats_System_PowerUseSummary)
}
if norm := normalizeRepeatedMessage(s1.GetScreenBrightness(), totalTimeHour); !norm.IsNil() {
s.ScreenBrightness = norm.Interface().([]*bspb.BatteryStats_System_ScreenBrightness)
}
if norm := normalizeMessage(s1.GetSignalScanningTime(), totalTimeHour); norm != nil {
s.SignalScanningTime = norm.(*bspb.BatteryStats_System_SignalScanningTime)
}
if norm := normalizeRepeatedMessage(s1.GetSignalStrength(), totalTimeHour); !norm.IsNil() {
s.SignalStrength = norm.Interface().([]*bspb.BatteryStats_System_SignalStrength)
}
if norm := normalizeRepeatedMessage(s1.GetWakeupReason(), totalTimeHour); !norm.IsNil() {
s.WakeupReason = norm.Interface().([]*bspb.BatteryStats_System_WakeupReason)
}
if norm := normalizeRepeatedMessage(s1.GetWifiState(), totalTimeHour); !norm.IsNil() {
s.WifiState = norm.Interface().([]*bspb.BatteryStats_System_WifiState)
}
if norm := normalizeRepeatedMessage(s1.GetWifiSupplicantState(), totalTimeHour); !norm.IsNil() {
s.WifiSupplicantState = norm.Interface().([]*bspb.BatteryStats_System_WifiSupplicantState)
}
p.System = s
p.App = normApp.App
return p, nil
}
// Normalize app data
func normalizeApp(a *bspb.BatteryStats_App, totalTimeHour float64) *bspb.BatteryStats_App {
if a == nil {
return nil
}
res := proto.Clone(a).(*bspb.BatteryStats_App)
if norm := normalizeMessage(a.GetForeground(), totalTimeHour); norm != nil {
res.Foreground = norm.(*bspb.BatteryStats_App_Foreground)
}
if norm := normalizeAppApk(a.GetApk(), totalTimeHour); norm != nil {
res.Apk = norm
}
normalizeAppChildren(res.GetChild(), totalTimeHour)
if norm := normalizeMessage(a.GetNetwork(), totalTimeHour); norm != nil {
res.Network = norm.(*bspb.BatteryStats_App_Network)
}
if norm := normalizeMessage(a.GetPowerUseItem(), totalTimeHour); norm != nil {
res.PowerUseItem = norm.(*bspb.BatteryStats_App_PowerUseItem)
}
if norm := normalizeRepeatedMessage(a.GetProcess(), totalTimeHour).Interface(); norm != nil {
res.Process = norm.([]*bspb.BatteryStats_App_Process)
}
if norm := normalizeRepeatedMessage(a.GetSensor(), totalTimeHour).Interface(); norm != nil {
res.Sensor = norm.([]*bspb.BatteryStats_App_Sensor)
}
if norm := normalizeMessage(a.GetStateTime(), totalTimeHour); norm != nil {
res.StateTime = norm.(*bspb.BatteryStats_App_StateTime)
}
if norm := normalizeMessage(a.GetVibrator(), totalTimeHour); norm != nil {
res.Vibrator = norm.(*bspb.BatteryStats_App_Vibrator)
}
if norm := normalizeRepeatedMessage(a.GetWakelock(), totalTimeHour).Interface(); norm != nil {
res.Wakelock = norm.([]*bspb.BatteryStats_App_Wakelock)
}
if norm := normalizeMessage(a.GetWifi(), totalTimeHour); norm != nil {
res.Wifi = norm.(*bspb.BatteryStats_App_Wifi)
}
if norm := normalizeRepeatedMessage(a.GetUserActivity(), totalTimeHour).Interface(); norm != nil {
res.UserActivity = norm.([]*bspb.BatteryStats_App_UserActivity)
}
if norm := normalizeRepeatedMessage(a.GetScheduledJob(), totalTimeHour).Interface(); norm != nil {
res.ScheduledJob = norm.([]*bspb.BatteryStats_App_ScheduledJob)
}
return res
}
// normalizeAppApk normalizes values in the "apk" section of App.
func normalizeAppApk(p *bspb.BatteryStats_App_Apk, totalTimeHour float64) *bspb.BatteryStats_App_Apk {
norm := &bspb.BatteryStats_App_Apk{}
// there's only one wakeups value per apk
norm.Wakeups = proto.Float32(float32(roundToTwoDecimal(float64(p.GetWakeups()) / totalTimeHour)))
norm.Service = normalizeRepeatedMessage(p.GetService(), totalTimeHour).Interface().([]*bspb.BatteryStats_App_Apk_Service)
return norm
}
// normalizeAppChildren normalizes values in the "child" section of App.
func normalizeAppChildren(children []*bspb.BatteryStats_App_Child, totalTimeHour float64) {
for _, c := range children {
c.Apk = normalizeAppApk(c.GetApk(), totalTimeHour)
}
}
// normalizeRepeatedMessage loops over a repeated message value and calls normalizeMessage
// for each if the message.
func normalizeRepeatedMessage(list interface{}, totalTimeHour float64) reflect.Value {
l := genericListOrDie(list)
out := reflect.MakeSlice(l.Type(), 0, l.Len())
for i := 0; i < l.Len(); i++ {
item := l.Index(i)
norm := normalizeMessage(item.Interface().(proto.Message), totalTimeHour)
out = reflect.Append(out, reflect.ValueOf(norm))
}
if out.Len() == 0 {
return reflect.Zero(l.Type())
}
return out
}
// normalizeMessage extracts the struct within the message and calls normalizeStruct
// to normalize the value contained.
func normalizeMessage(p proto.Message, totalTimeHour float64) proto.Message {
in := reflect.ValueOf(p)
if in.IsNil() {
return nil
}
out := reflect.New(in.Type().Elem())
normalizeStruct(out.Elem(), in.Elem(), totalTimeHour, idMap[reflect.TypeOf(p)])
return out.Interface().(proto.Message)
}
// normalizeStruct traverses a struct value and normalizes each field
func normalizeStruct(out, in reflect.Value, totalTimeHour float64, ids map[int]bool) {
for i := 0; i < in.NumField()-1; i++ {
fieldPtrV := in.Field(i)
if fieldPtrV.IsNil() {
continue
}
normV := reflect.New(fieldPtrV.Type().Elem())
normalizeValue(normV.Elem(), fieldPtrV.Elem(), in.String(), totalTimeHour, ids[i])
out.Field(i).Set(normV)
}
}
// normalizeValue normalizes numerical values.
func normalizeValue(out, in reflect.Value, section string, totalTimeHour float64, id bool) {
switch in.Kind() {
case reflect.Float32, reflect.Float64:
if id {
out.Set(in)
} else {
out.SetFloat(roundToTwoDecimal(in.Float() / totalTimeHour))
}
case reflect.Int32, reflect.Int64:
if id {
out.Set(in)
} else {
// Some rounding error is okay for the integer fields.
out.SetInt(int64(float64(in.Int()) / totalTimeHour))
}
case reflect.String:
if !id {
reportWarningf("Tried to normalize a string for %s", section)
}
out.Set(in)
default:
reportWarningf("Normalizing %s type in %s not supported", in.Kind().String(), section)
}
}
此差异已折叠。
// Copyright 2015 Google Inc. All Rights Reserved.
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
......@@ -16,6 +16,12 @@
// that are used in parsing of checkin format.
package checkinutil
import (
"encoding/csv"
"fmt"
"strings"
)
// ChildInfo contains linkage information for App.Child.
type ChildInfo struct {
// predefined parent name (e.g., GOOGLE_SERVICES for gms and gsf)
......@@ -37,6 +43,7 @@ type CheckinReport struct {
Radio string
Bootloader string
SDKVersion int32
IsAltMode bool // True if the android wear device is paired to an ALT mode companion
CellOperator string
CountryCode string
RawBatteryStats [][]string
......@@ -64,5 +71,27 @@ type PrefixCounter struct {
// Count increments the named counter by inc.
func (c *PrefixCounter) Count(name string, inc int) {
c.Counter.Count(c.Prefix+"-"+name, inc)
// Replace null character in counter name by "null" to avoid the "Counter name contains embedded
// null" error.
counterName := strings.Replace(c.Prefix+"-"+name, "\x00", "null", -1)
c.Counter.Count(counterName, inc)
}
// ParseCSV parses the content of a CSV file into a two-dimensional slice of strings.
func ParseCSV(content string) [][]string {
reader := csv.NewReader(strings.NewReader(content))
reader.FieldsPerRecord = -1 // allow a variable number of fields
reader.LazyQuotes = true // A bug report might include bare quotes
reader.TrimLeadingSpace = true
records, err := reader.ReadAll()
if err != nil {
fmt.Println(err)
return nil
}
for i := range records {
for j := range records[i] {
records[i][j] = strings.TrimSpace(records[i][j])
}
}
return records
}
// Copyright 2015 Google Inc. All Rights Reserved.
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
......@@ -27,23 +27,38 @@ import (
var (
optimized = flag.Bool("optimized", true, "Whether to output optimized js files. Disable for local debugging.")
port = flag.Int("port", 9999, "service port")
scriptsDir = flag.String("scripts_dir", "./scripts", "Directory containing Historian and kernel trace Python scripts.")
templateDir = flag.String("template_dir", "./templates", "Directory containing HTML templates.")
// resVersion should be incremented whenever the JS or CSS files are modified.
resVersion = flag.Int("res_version", 0, "The current version of JS and CSS files. Used to force JS and CSS reloading to avoid cache issues when rolling out new versions.")
)
type analysisServer struct{}
func (*analysisServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
log.Printf("Starting processing for: %s", r.Method)
func (s *analysisServer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
log.Printf("Trace starting analysisServer processing for: %s", r.Method)
defer log.Printf("Trace finished analysisServer processing for: %s", r.Method)
analyzer.UploadHandler(w, r)
switch r.Method {
case "GET":
analyzer.UploadHandler(w, r)
case "POST":
r.ParseForm()
analyzer.HTTPAnalyzeHandler(w, r)
default:
http.Error(w, fmt.Sprintf("Method %s not allowed", r.Method), http.StatusMethodNotAllowed)
}
}
func initFrontend() {
http.HandleFunc("/", analyzer.UploadHandler)
http.Handle("/", &analysisServer{})
http.Handle("/static/", http.FileServer(http.Dir(".")))
http.Handle("/compiled/", http.FileServer(http.Dir(".")))
http.Handle("/third_party/", http.FileServer(http.Dir(".")))
if *optimized == false {
http.Handle("/third_party/", http.FileServer(http.Dir(".")))
http.Handle("/js/", http.FileServer(http.Dir(".")))
}
}
......@@ -52,7 +67,9 @@ func main() {
flag.Parse()
initFrontend()
analyzer.InitTemplates()
analyzer.InitTemplates(*templateDir)
analyzer.SetScriptsDir(*scriptsDir)
analyzer.SetResVersion(*resVersion)
analyzer.SetIsOptimized(*optimized)
log.Println("Listening on port: ", *port)
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), nil))
......
此差异已折叠。
// Copyright 2015 Google Inc. All Rights Reserved.
// Copyright 2016 Google Inc. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
......@@ -12,6 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
// local_checkin_parse parses checkin format of batterystats into a batterystats proto.
package main
import (
......@@ -19,9 +20,11 @@ import (
"fmt"
"io/ioutil"
"log"
"strings"
"time"
"github.com/golang/protobuf/proto"
"github.com/google/battery-historian/bugreportutils"
"github.com/google/battery-historian/checkinparse"
"github.com/google/battery-historian/checkinutil"
"github.com/google/battery-historian/packageutils"
......@@ -47,8 +50,16 @@ func main() {
log.Fatalf("Cannot open the file %s: %v", *inputFile, err)
}
br := string(c)
s := &sessionpb.Checkin{Checkin: proto.String(br)}
br, fname, err := bugreportutils.ExtractBugReport(*inputFile, c)
if err != nil {
log.Fatalf("Error getting file contents: %v", err)
}
fmt.Printf("Parsing %s\n", fname)
bs := bugreportutils.ExtractBatterystatsCheckin(br)
if strings.Contains(bs, "Exception occurred while dumping") {
log.Fatalf("Exception found in battery dump.")
}
s := &sessionpb.Checkin{Checkin: proto.String(bs)}
pkgs, errs := packageutils.ExtractAppsFromBugReport(br)
if len(errs) > 0 {
log.Fatalf("Errors encountered when getting package list: %v", errs)
......
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
......@@ -44,5 +44,4 @@ message Build {
// Tags, e.g. "dev-keys"
repeated string tags = 9;
}
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
screenshots/app.png

187.4 KB | W: | H:

screenshots/app.png

318.9 KB | W: | H:

screenshots/app.png
screenshots/app.png
screenshots/app.png
screenshots/app.png
  • 2-up
  • Swipe
  • Onion skin
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册