提交 6a6b77a1 编写于 作者: S Sharif Elgamal

Merge branch 'master' of github.com:kubernetes/minikube into m2

# Release Notes
## Version 1.9.0-beta.1 - 2020-03-18
New features
* Use Kubernetes v1.18.0-rc.1 by default [#7076](https://github.com/kubernetes/minikube/pull/7076)
* Upgrade Docker driver to preferred (Linux), default on other platforms [#7090](https://github.com/kubernetes/minikube/pull/7090)
* Upgrade Docker, from 19.03.7 to 19.03.8 [#7040](https://github.com/kubernetes/minikube/pull/7040)
* Upgrade Docker, from 19.03.6 to 19.03.7 [#6939](https://github.com/kubernetes/minikube/pull/6939)
* Upgrade dashboard to v2.0.0-rc6 [#7098](https://github.com/kubernetes/minikube/pull/7098)
* Upgrade crio to 1.17.1 [#7099](https://github.com/kubernetes/minikube/pull/7099)
* Updated French translation [#7055](https://github.com/kubernetes/minikube/pull/7055)
Bugfixes
* If user doesn't specify driver, don't validate against existing cluster [#7096](https://github.com/kubernetes/minikube/pull/7096)
* Strip the version prefix before calling semver [#7054](https://github.com/kubernetes/minikube/pull/7054)
* Move some of the driver validation before driver selection [#7080](https://github.com/kubernetes/minikube/pull/7080)
* Fix bug where global config memory was ignored [#7082](https://github.com/kubernetes/minikube/pull/7082)
* Remove controllerManager from the kubeadm v1beta2 template [#7030](https://github.com/kubernetes/minikube/pull/7030)
* Delete: output underlying status failure [#7043](https://github.com/kubernetes/minikube/pull/7043)
* status: error properly if cluster does not exist [#7041](https://github.com/kubernetes/minikube/pull/7041)
Huge thank you for this release towards our contributors:
- Anders F Björklund
- Medya Ghazizadeh
- Priya Wadhwa
- RA489
- Richard Wall
- Sharif Elgamal
- Thomas Strömberg
- Vikky Omkar
- jumahmohammad
## Version 1.8.2 - 2020-03-13
Shiny new improvements:
......
......@@ -14,18 +14,16 @@
# Bump these on release - and please check ISO_VERSION for correctness.
VERSION_MAJOR ?= 1
VERSION_MINOR ?= 8
VERSION_BUILD ?= 2
RAW_VERSION=$(VERSION_MAJOR).$(VERSION_MINOR).${VERSION_BUILD}
VERSION_MINOR ?= 9
VERSION_BUILD ?= 0-beta.1
RAW_VERSION=$(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_BUILD)
VERSION ?= v$(RAW_VERSION)
KUBERNETES_VERSION ?= $(shell egrep "DefaultKubernetesVersion =" pkg/minikube/constants/constants.go | cut -d \" -f2)
KIC_VERSION ?= $(shell egrep "Version =" pkg/drivers/kic/types.go | cut -d \" -f2)
PRELOADED_TARBALL_VERSION ?= $(shell egrep "PreloadVersion =" pkg/minikube/download/preload.go | cut -d \" -f2)
PRELOADED_VOLUMES_GCS_BUCKET ?= $(shell egrep "PreloadBucket =" pkg/minikube/download/preload.go | cut -d \" -f2)
# Default to .0 for higher cache hit rates, as build increments typically don't require new ISO versions
ISO_VERSION ?= v$(VERSION_MAJOR).$(VERSION_MINOR).0
ISO_VERSION ?= v$(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_BUILD)
# Dashes are valid in semver, but not Linux packaging. Use ~ to delimit alpha/beta
DEB_VERSION ?= $(subst -,~,$(RAW_VERSION))
RPM_VERSION ?= $(DEB_VERSION)
......@@ -34,7 +32,7 @@ RPM_VERSION ?= $(DEB_VERSION)
GO_VERSION ?= 1.13.8
INSTALL_SIZE ?= $(shell du out/minikube-windows-amd64.exe | cut -f1)
BUILDROOT_BRANCH ?= 2019.02.9
BUILDROOT_BRANCH ?= 2019.02.10
REGISTRY?=gcr.io/k8s-minikube
# Get git commit id
......@@ -54,7 +52,7 @@ MINIKUBE_BUCKET ?= minikube/releases
MINIKUBE_UPLOAD_LOCATION := gs://${MINIKUBE_BUCKET}
MINIKUBE_RELEASES_URL=https://github.com/kubernetes/minikube/releases/download
KERNEL_VERSION ?= 4.19.94
KERNEL_VERSION ?= 4.19.107
# latest from https://github.com/golangci/golangci-lint/releases
GOLINT_VERSION ?= v1.23.6
# Limit number of default jobs, to avoid the CI builds running out of memory
......@@ -526,14 +524,8 @@ kic-base-image: ## builds the base image used for kic.
docker build -f ./hack/images/kicbase.Dockerfile -t $(REGISTRY)/kicbase:$(KIC_VERSION)-snapshot --build-arg COMMIT_SHA=${VERSION}-$(COMMIT) --target base .
.PHONY: upload-preloaded-images-tar
upload-preloaded-images-tar: generate-preloaded-images-tar # Upload the preloaded images tar to the GCS bucket. Specify a specific kubernetes version to build via `KUBERNETES_VERSION=vx.y.z make upload-preloaded-images-tar`.
gsutil cp out/preloaded-images-k8s-${PRELOADED_TARBALL_VERSION}-${KUBERNETES_VERSION}-docker-overlay2.tar.lz4 gs://${PRELOADED_VOLUMES_GCS_BUCKET}
gsutil acl ch -u AllUsers:R gs://${PRELOADED_VOLUMES_GCS_BUCKET}/preloaded-images-k8s-${PRELOADED_TARBALL_VERSION}-${KUBERNETES_VERSION}-docker-overlay2.tar.lz4
.PHONY: generate-preloaded-images-tar
generate-preloaded-images-tar:
go run ./hack/preload-images/preload_images.go -kubernetes-version ${KUBERNETES_VERSION} -preloaded-tarball-version ${PRELOADED_TARBALL_VERSION}
upload-preloaded-images-tar: out/minikube # Upload the preloaded images for oldest supported, newest supported, and default kubernetes versions to GCS.
go run ./hack/preload-images/*.go
.PHONY: push-storage-provisioner-image
push-storage-provisioner-image: storage-provisioner-image ## Push storage-provisioner docker image using gcloud
......
......@@ -32,7 +32,7 @@ import (
const (
// number of problems per log to output
numberOfProblems = 5
numberOfProblems = 10
)
var (
......
......@@ -296,6 +296,7 @@ func runStart(cmd *cobra.Command, args []string) {
exit.WithCodeT(exit.Data, "Unable to load config: {{.error}}", out.V{"error": err})
}
validateSpecifiedDriver(existing)
ds := selectDriver(existing)
driverName := ds.Name
glog.Infof("selected driver: %s", driverName)
......@@ -472,8 +473,18 @@ func selectDriver(existing *config.ClusterConfig) registry.DriverState {
}
// Default to looking at the new driver parameter
if viper.GetString("driver") != "" {
ds := driver.Status(viper.GetString("driver"))
if d := viper.GetString("driver"); d != "" {
if vmd := viper.GetString("vm-driver"); vmd != "" {
// Output a warning
warning := `Both driver={{.driver}} and vm-driver={{.vmd}} have been set.
Since vm-driver is deprecated, minikube will default to driver={{.driver}}.
If vm-driver is set in the global config, please run "minikube config unset vm-driver" to resolve this warning.
`
out.T(out.Warning, warning, out.V{"driver": d, "vmd": vmd})
}
ds := driver.Status(d)
out.T(out.Sparkle, `Using the {{.driver}} driver based on user configuration`, out.V{"driver": ds.String()})
return ds
}
......@@ -502,36 +513,24 @@ func selectDriver(existing *config.ClusterConfig) registry.DriverState {
return pick
}
// validateDriver validates that the selected driver appears sane, exits if not
func validateDriver(ds registry.DriverState, existing *config.ClusterConfig) {
name := ds.Name
glog.Infof("validating driver %q against %+v", name, existing)
if !driver.Supported(name) {
exit.WithCodeT(exit.Unavailable, "The driver {{.experimental}} '{{.driver}}' is not supported on {{.os}}", out.V{"driver": name, "os": runtime.GOOS})
// validateSpecifiedDriver makes sure that if a user has passed in a driver
// it matches the existing cluster if there is one
func validateSpecifiedDriver(existing *config.ClusterConfig) {
if existing == nil {
return
}
st := ds.State
glog.Infof("status for %s: %+v", name, st)
if st.Error != nil {
out.ErrLn("")
out.WarningT("'{{.driver}}' driver reported an issue: {{.error}}", out.V{"driver": name, "error": st.Error})
out.ErrT(out.Tip, "Suggestion: {{.fix}}", out.V{"fix": translate.T(st.Fix)})
if st.Doc != "" {
out.ErrT(out.Documentation, "Documentation: {{.url}}", out.V{"url": st.Doc})
}
out.ErrLn("")
if !st.Installed && !viper.GetBool(force) {
if existing != nil && name == existing.Driver {
exit.WithCodeT(exit.Unavailable, "{{.driver}} does not appear to be installed, but is specified by an existing profile. Please run 'minikube delete' or install {{.driver}}", out.V{"driver": name})
}
exit.WithCodeT(exit.Unavailable, "{{.driver}} does not appear to be installed", out.V{"driver": name})
}
old := existing.Driver
var requested string
if d := viper.GetString("driver"); d != "" {
requested = d
} else if d := viper.GetString("vm-driver"); d != "" {
requested = d
}
if existing == nil {
// Neither --vm-driver or --driver was specified
if requested == "" {
return
}
if old == requested {
return
}
......@@ -545,7 +544,6 @@ func validateDriver(ds registry.DriverState, existing *config.ClusterConfig) {
if err != nil {
exit.WithError("Error getting primary cp", err)
}
machineName := driver.MachineName(*existing, cp)
h, err := api.Load(machineName)
if err != nil {
......@@ -553,25 +551,51 @@ func validateDriver(ds registry.DriverState, existing *config.ClusterConfig) {
return
}
if h.Driver.DriverName() == name {
return
}
out.ErrT(out.Conflict, `The existing "{{.profile_name}}" VM that was created using the "{{.old_driver}}" driver, and is incompatible with the "{{.driver}}" driver.`,
out.V{"profile_name": machineName, "driver": name, "old_driver": h.Driver.DriverName()})
out.ErrT(out.Conflict, `The existing "{{.profile_name}}" VM was created using the "{{.old_driver}}" driver, and is incompatible with the "{{.driver}}" driver.`,
out.V{"profile_name": machineName, "driver": requested, "old_driver": h.Driver.DriverName()})
out.ErrT(out.Workaround, `To proceed, either:
1) Delete the existing "{{.profile_name}}" cluster using: '{{.command}} delete'
1) Delete the existing "{{.profile_name}}" cluster using: '{{.command}} delete'
* or *
* or *
2) Start the existing "{{.profile_name}}" cluster using: '{{.command}} start --driver={{.old_driver}}'
`, out.V{"command": minikubeCmd(), "old_driver": h.Driver.DriverName(), "profile_name": machineName})
2) Start the existing "{{.profile_name}}" cluster using: '{{.command}} start --driver={{.old_driver}}'
`, out.V{"command": minikubeCmd(), "old_driver": h.Driver.DriverName(), "profile_name": machineName})
exit.WithCodeT(exit.Config, "Exiting.")
}
// validateDriver validates that the selected driver appears sane, exits if not
func validateDriver(ds registry.DriverState, existing *config.ClusterConfig) {
name := ds.Name
glog.Infof("validating driver %q against %+v", name, existing)
if !driver.Supported(name) {
exit.WithCodeT(exit.Unavailable, "The driver {{.experimental}} '{{.driver}}' is not supported on {{.os}}", out.V{"driver": name, "os": runtime.GOOS})
}
st := ds.State
glog.Infof("status for %s: %+v", name, st)
if st.Error != nil {
out.ErrLn("")
out.WarningT("'{{.driver}}' driver reported an issue: {{.error}}", out.V{"driver": name, "error": st.Error})
out.ErrT(out.Tip, "Suggestion: {{.fix}}", out.V{"fix": translate.T(st.Fix)})
if st.Doc != "" {
out.ErrT(out.Documentation, "Documentation: {{.url}}", out.V{"url": st.Doc})
}
out.ErrLn("")
if !st.Installed && !viper.GetBool(force) {
if existing != nil && name == existing.Driver {
exit.WithCodeT(exit.Unavailable, "{{.driver}} does not appear to be installed, but is specified by an existing profile. Please run 'minikube delete' or install {{.driver}}", out.V{"driver": name})
}
exit.WithCodeT(exit.Unavailable, "{{.driver}} does not appear to be installed", out.V{"driver": name})
}
}
}
func selectImageRepository(mirrorCountry string, v semver.Version) (bool, string, error) {
var tryCountries []string
var fallback string
......@@ -686,6 +710,9 @@ func memoryLimits(drvName string) (int, int, error) {
// suggestMemoryAllocation calculates the default memory footprint in MB
func suggestMemoryAllocation(sysLimit int, containerLimit int) int {
if mem := viper.GetInt(memory); mem != 0 {
return mem
}
fallback := 2200
maximum := 6000
......@@ -842,7 +869,7 @@ func generateCfgFromFlags(cmd *cobra.Command, k8sVersion string, drvName string)
repository := viper.GetString(imageRepository)
mirrorCountry := strings.ToLower(viper.GetString(imageMirrorCountry))
if strings.ToLower(repository) == "auto" || mirrorCountry != "" {
found, autoSelectedRepository, err := selectImageRepository(mirrorCountry, semver.MustParse(k8sVersion))
found, autoSelectedRepository, err := selectImageRepository(mirrorCountry, semver.MustParse(strings.TrimPrefix(k8sVersion, version.VersionPrefix)))
if err != nil {
exit.WithError("Failed to check main repository and mirrors for images for images", err)
}
......
......@@ -70,6 +70,50 @@ func TestGetKuberneterVersion(t *testing.T) {
}
}
func TestMirrorCountry(t *testing.T) {
// Set default disk size value in lieu of flag init
viper.SetDefault(humanReadableDiskSize, defaultDiskSize)
k8sVersion := constants.DefaultKubernetesVersion
var tests = []struct {
description string
k8sVersion string
imageRepository string
mirrorCountry string
cfg *cfg.ClusterConfig
}{
{
description: "image-repository none, image-mirror-country none",
imageRepository: "",
mirrorCountry: "",
},
{
description: "image-repository auto, image-mirror-country none",
imageRepository: "auto",
mirrorCountry: "",
},
{
description: "image-repository auto, image-mirror-country china",
imageRepository: "auto",
mirrorCountry: "cn",
},
}
for _, test := range tests {
t.Run(test.description, func(t *testing.T) {
cmd := &cobra.Command{}
viper.SetDefault(imageRepository, test.imageRepository)
viper.SetDefault(imageMirrorCountry, test.mirrorCountry)
config, _, err := generateCfgFromFlags(cmd, k8sVersion, "none")
if err != nil {
t.Fatalf("Got unexpected error %v during config generation", err)
}
// the result can still be "", but anyway
_ = config.KubernetesConfig.ImageRepository
})
}
}
func TestGenerateCfgFromFlagsHTTPProxyHandling(t *testing.T) {
// Set default disk size value in lieu of flag init
viper.SetDefault(humanReadableDiskSize, defaultDiskSize)
......
......@@ -90,7 +90,7 @@ spec:
containers:
- name: kubernetes-dashboard
# WARNING: This must match pkg/minikube/bootstrapper/images/images.go
image: kubernetesui/dashboard:v2.0.0-beta8
image: kubernetesui/dashboard:v2.0.0-rc6
ports:
- containerPort: 9090
protocol: TCP
......
# Minikube Registry Aliases Addon
An addon to minikube that can help push and pull from the minikube registry using custom domain names. The custom domain names will be made resolveable from with in cluster and at minikube node.
## How to use ?
### Start minikube
```shell
minikube start -p demo
```
This addon depends on `registry` addon, it need to be enabled before the alias addon is installed:
### Enable internal registry
```shell
minikube addons enable registry
```
Verifying the registry deployment
```shell
watch kubectl get pods -n kube-system
```
```shell
NAME READY STATUS RESTARTS AGE
coredns-6955765f44-kpbzt 1/1 Running 0 16m
coredns-6955765f44-lzlsv 1/1 Running 0 16m
etcd-demo 1/1 Running 0 16m
kube-apiserver-demo 1/1 Running 0 16m
kube-controller-manager-demo 1/1 Running 0 16m
kube-proxy-q8rb9 1/1 Running 0 16m
kube-scheduler-demo 1/1 Running 0 16m
*registry-4k8zs* 1/1 Running 0 40s
registry-proxy-vs8jt 1/1 Running 0 40s
storage-provisioner 1/1 Running 0 16m
```
```shell
kubectl get svc -n kube-system
```
```shell
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 17m
registry ClusterIP 10.97.247.75 <none> 80/TCP 94s
```
>
> **NOTE:**
> Please make a note of the CLUSTER-IP of `registry` service
### Enable registry aliases addon
```shell
minikube addons enable registry-aliases
🌟 The 'registry-aliases' addon is enabled
```
You can check the mikikube vm's `/etc/hosts` file for the registry aliases entries:
```shell
watch minikube ssh -- cat /etc/hosts
```
```shell
127.0.0.1 localhost
127.0.1.1 demo
10.97.247.75 example.org
10.97.247.75 example.com
10.97.247.75 test.com
10.97.247.75 test.org
```
The above output shows that the Daemonset has added the `registryAliases` from the ConfigMap pointing to the internal registry's __CLUSTER-IP__.
### Update CoreDNS
The coreDNS would have been automatically updated by the patch-coredns. A successful job run will have coredns ConfigMap updated like:
```yaml
apiVersion: v1
data:
Corefile: |-
.:53 {
errors
health
rewrite name example.com registry.kube-system.svc.cluster.local
rewrite name example.org registry.kube-system.svc.cluster.local
rewrite name test.com registry.kube-system.svc.cluster.local
rewrite name test.org registry.kube-system.svc.cluster.local
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
proxy . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
kind: ConfigMap
metadata:
name: coredns
```
To verify it run the following command:
```shell
kubectl get cm -n kube-system coredns -o yaml
```
Once you have successfully patched you can now push and pull from the registry using suffix `example.com`, `example.org`,`test.com` and `test.org`.
The successful run will show the following extra pods (Daemonset, Job) in `kube-system` namespace:
```shell
NAME READY STATUS RESTARTS AGE
registry-aliases-hosts-update-995vx 1/1 Running 0 47s
registry-aliases-patch-core-dns-zsxfc 0/1 Completed 0 47s
```
## Verify with sample application
You can verify the deployment end to end using the example [application](https://github.com/kameshsampath/minikube-registry-aliases-demo).
```shell
git clone https://github.com/kameshsampath/minikube-registry-aliases-demo
cd minikube-registry-aliases-demo
```
Make sure you set the docker context using `eval $(minikube -p demo docker-env)`
Deploy the application using [Skaffold](https://skaffold.dev):
```shell
skaffold dev --port-forward
```
Once the application is running try doing `curl localhost:8080` to see the `Hello World` response
You can also update [skaffold.yaml](./skaffold.yaml) and [app.yaml](.k8s/app.yaml), to use `test.org`, `test.com` or `example.org` as container registry urls, and see all the container image names resolves to internal registry, resulting in successful build and deployment.
> **NOTE**:
>
> You can also update [skaffold.yaml](./skaffold.yaml) and [app. yaml](.k8s/app.yaml), to use `test.org`, `test.com` or > `example.org` as container registry urls, and see all the > container image names resolves to internal registry, resulting in successful build and deployment.
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: registry-aliases-hosts-update
namespace: kube-system
labels:
kubernetes.io/minikube-addons: registry-aliases
addonmanager.kubernetes.io/mode: Reconcile
spec:
selector:
matchLabels:
app: registry-aliases-hosts-update
template:
metadata:
labels:
app: registry-aliases-hosts-update
spec:
initContainers:
- name: update
image: registry.fedoraproject.org/fedora
volumeMounts:
- name: etchosts
mountPath: /host-etc/hosts
readOnly: false
env:
- name: REGISTRY_ALIASES
valueFrom:
configMapKeyRef:
name: registry-aliases
key: registryAliases
command:
- bash
- -ce
- |
NL=$'\n'
TAB=$'\t'
HOSTS="$(cat /host-etc/hosts)"
[ -z "$REGISTRY_SERVICE_HOST" ] && echo "Failed to get hosts entry for default registry" && exit 1;
for H in $REGISTRY_ALIASES; do
echo "$HOSTS" | grep "$H" || HOSTS="$HOSTS$NL$REGISTRY_SERVICE_HOST$TAB$H";
done;
echo "$HOSTS" | diff -u /host-etc/hosts - || echo "$HOSTS" > /host-etc/hosts
echo "Done."
containers:
- name: pause-for-update
image: gcr.io/google_containers/pause-amd64:3.1
terminationGracePeriodSeconds: 30
volumes:
- name: etchosts
hostPath:
path: /etc/hosts
---
apiVersion: batch/v1
kind: Job
metadata:
name: registry-aliases-patch-core-dns
namespace: kube-system
spec:
ttlSecondsAfterFinished: 100
template:
spec:
serviceAccountName: registry-aliases-sa
volumes:
- name: minikube
hostPath:
path: /var/lib/minikube/binaries
containers:
- name: core-dns-patcher
image: quay.io/rhdevelopers/core-dns-patcher
imagePullPolicy: IfNotPresent
# using the kubectl from the minikube instance
volumeMounts:
- mountPath: /var/lib/minikube/binaries
name: minikube
readOnly: true
restartPolicy: Never
backoffLimit: 4
\ No newline at end of file
apiVersion: v1
kind: ConfigMap
metadata:
name: registry-aliases
namespace: kube-system
labels:
kubernetes.io/minikube-addons: registry-aliases
addonmanager.kubernetes.io/mode: Reconcile
data:
# Add additonal hosts seperated by new-line
registryAliases: >-
example.org
example.com
test.com
test.org
# default registry address in minikube when enabled via minikube addons enable registry
registrySvc: registry.kube-system.svc.cluster.local
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: registry-aliases-crb
subjects:
- kind: ServiceAccount
name: registry-aliases-sa
namespace: kube-system
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
\ No newline at end of file
apiVersion: v1
kind: ServiceAccount
metadata:
name: registry-aliases-sa
namespace: kube-system
\ No newline at end of file
......@@ -11,3 +11,4 @@ sha256 70d4c746fe207422c78420dc4239768f485eea639a38c993c02872ec6305dd1d v1.15.2.
sha256 05f9614c4d5970b4662499b84c270b0ab953596ee863dcd09c9dc7a2d2f09789 v1.16.0.tar.gz
sha256 57e1ee990ef2d5af8b32c33a21b4998682608e3556dcf1d3349666f55e7d95b9 v1.16.1.tar.gz
sha256 23a797762e4544ee7c171ef138cfc1141a3f0acc2838d9965c2a58e53b16c3ae v1.17.0.tar.gz
sha256 7967e9218fdfb59d6005a9e19c1668469bc5566c2a35927cffe7de8656bb22c7 v1.17.1.tar.gz
......@@ -4,8 +4,8 @@
#
################################################################################
CRIO_BIN_VERSION = v1.17.0
CRIO_BIN_COMMIT = 6d0ffae63b9b7d8f07e7f9cf50736a67fb31faf3
CRIO_BIN_VERSION = v1.17.1
CRIO_BIN_COMMIT = ee2de87bd8e2a7a84799476cb4fc4ce8a78fdf6d
CRIO_BIN_SITE = https://github.com/cri-o/cri-o/archive
CRIO_BIN_SOURCE = $(CRIO_BIN_VERSION).tar.gz
CRIO_BIN_DEPENDENCIES = host-go libgpgme
......
......@@ -27,6 +27,7 @@ require (
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
github.com/google/go-cmp v0.3.1
github.com/google/go-containerregistry v0.0.0-20200131185320-aec8da010de2
github.com/google/go-github v17.0.0+incompatible
github.com/googleapis/gnostic v0.3.0 // indirect
github.com/hashicorp/go-getter v1.4.0
github.com/hashicorp/go-retryablehttp v0.5.4
......
......@@ -68,6 +68,7 @@ github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYU
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a h1:idn718Q4B6AGu/h5Sxe66HYVdqdGu2l9Iebqhi/AEoA=
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/auth0/go-jwt-middleware v0.0.0-20170425171159-5493cabe49f7/go.mod h1:LWMyo4iOLWXHGdBki7NIht1kHru/0wM179h+d3g8ATM=
github.com/aws/aws-sdk-go v1.15.78 h1:LaXy6lWR0YK7LKyuU0QWy2ws/LWTPfYV/UgfiBu4tvY=
......@@ -93,6 +94,7 @@ github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kB
github.com/bifurcation/mint v0.0.0-20180715133206-93c51c6ce115/go.mod h1:zVt7zX3K/aDCk9Tj+VM7YymsX66ERvzCJzw8rFCX2JU=
github.com/blang/semver v3.5.0+incompatible h1:CGxCgetQ64DKk7rdZ++Vfnb1+ogGNnB17OJKJXD2Cfs=
github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/boltdb/bolt v1.3.1 h1:JQmyP4ZBrce+ZQu0dY660FMfatumYDLun9hBCUVIkF4=
github.com/boltdb/bolt v1.3.1/go.mod h1:clJnj/oiGkjum5o1McbSZDSLxVThjynRyGBgiAx27Ps=
github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625/go.mod h1:HYsPBTaaSFSlLx/70C2HPIMNZpVV8+vt/A+FMnYP11g=
github.com/c4milo/gotoolkit v0.0.0-20170318115440-bcc06269efa9 h1:+ziP/wVJWuAORkjv7386TRidVKY57X0bXBZFMeFlW+U=
......@@ -260,6 +262,7 @@ github.com/go-openapi/swag v0.19.5/go.mod h1:POnQmlKehdgb5mhVOsnJFsivZCEZ/vjK9gh
github.com/go-openapi/validate v0.18.0/go.mod h1:Uh4HdOzKt19xGIGm1qHf/ofbX1YQ4Y+MYsct2VUrAJ4=
github.com/go-openapi/validate v0.19.2/go.mod h1:1tRCw7m3jtI8eNWEEliiAqUIcBztB2KDnRCRMUi7GTA=
github.com/go-openapi/validate v0.19.5/go.mod h1:8DJv2CVJQ6kGNpFW6eV9N3JviE1C85nY1c2z52x1Gk4=
github.com/go-ozzo/ozzo-validation v3.5.0+incompatible h1:sUy/in/P6askYr16XJgTKq/0SZhiWsdg4WZGaLsGQkM=
github.com/go-ozzo/ozzo-validation v3.5.0+incompatible/go.mod h1:gsEKFIVnabGBt6mXmxK0MoFy+cZoTJY6mu5Ll3LVLBU=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/go-toolsmith/astcast v1.0.0/go.mod h1:mt2OdQTeAQcY4DQgPSArJjHCcOwlX+Wl/kwN+LbLGQ4=
......@@ -391,7 +394,9 @@ github.com/hashicorp/golang-lru v0.5.3/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uG
github.com/hashicorp/hcl v0.0.0-20180404174102-ef8a98b0bbce/go.mod h1:oZtUIOe8dh44I2q6ScRibXws4Ajl+d+nod3AaR9vL5w=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/heketi/heketi v9.0.1-0.20190917153846-c2e2a4ab7ab9+incompatible h1:ysqc8k973k1lLJ4BOOHAkx14K2nt4cLjsIm+hwWDZDE=
github.com/heketi/heketi v9.0.1-0.20190917153846-c2e2a4ab7ab9+incompatible/go.mod h1:bB9ly3RchcQqsQ9CpyaQwvva7RS5ytVoSoholZQON6o=
github.com/heketi/tests v0.0.0-20151005000721-f3775cbcefd6 h1:oJ/NLadJn5HoxvonA6VxG31lg0d6XOURNA09BTtM4fY=
github.com/heketi/tests v0.0.0-20151005000721-f3775cbcefd6/go.mod h1:xGMAM8JLi7UkZt1i4FQeQy0R2T8GLUwQhOP5M1gBhy4=
github.com/hooklift/assert v0.0.0-20170704181755-9d1defd6d214 h1:WgfvpuKg42WVLkxNwzfFraXkTXPK36bMqXvMFN67clI=
github.com/hooklift/assert v0.0.0-20170704181755-9d1defd6d214/go.mod h1:kj6hFWqfwSjFjLnYW5PK1DoxZ4O0uapwHRmd9jhln4E=
......
#!/bin/bash
# Copyright 2016 The Kubernetes Authors All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This script builds all the minikube binary for all 3 platforms as well as Windows-installer and .deb
# This is intended to be run on a new release tag in order to build/upload the required files for a release
set -eux -o pipefail
# Make sure the right golang version is installed based on Makefile
WANT_GOLANG_VERSION=$(grep '^GO_VERSION' Makefile | awk '{ print $3 }')
./hack/jenkins/installers/check_install_golang.sh $WANT_GOLANG_VERSION /usr/local
make upload-preloaded-images-tar
make clean
/*
Copyright 2020 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"github.com/pkg/errors"
"k8s.io/minikube/pkg/drivers/kic"
"k8s.io/minikube/pkg/drivers/kic/oci"
"k8s.io/minikube/pkg/minikube/bootstrapper/bsutil"
"k8s.io/minikube/pkg/minikube/bootstrapper/images"
"k8s.io/minikube/pkg/minikube/command"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/driver"
"k8s.io/minikube/pkg/minikube/localpath"
)
func generateTarball(kubernetesVersion, tarballFilename string) error {
defer func() {
if err := deleteMinikube(); err != nil {
fmt.Println(err)
}
}()
driver := kic.NewDriver(kic.Config{
KubernetesVersion: kubernetesVersion,
ContainerRuntime: driver.Docker,
OCIBinary: oci.Docker,
MachineName: profile,
ImageDigest: kic.BaseImage,
StorePath: localpath.MiniPath(),
CPU: 2,
Memory: 4000,
APIServerPort: 8080,
})
baseDir := filepath.Dir(driver.GetSSHKeyPath())
defer os.Remove(baseDir)
if err := os.MkdirAll(baseDir, 0755); err != nil {
return errors.Wrap(err, "mkdir")
}
if err := driver.Create(); err != nil {
return errors.Wrap(err, "creating kic driver")
}
// Now, get images to pull
imgs, err := images.Kubeadm("", kubernetesVersion)
if err != nil {
return errors.Wrap(err, "kubeadm images")
}
for _, img := range append(imgs, kic.OverlayImage) {
cmd := exec.Command("docker", "exec", profile, "docker", "pull", img)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "downloading %s", img)
}
}
// Transfer in k8s binaries
kcfg := config.KubernetesConfig{
KubernetesVersion: kubernetesVersion,
}
runner := command.NewKICRunner(profile, driver.OCIBinary)
if err := bsutil.TransferBinaries(kcfg, runner); err != nil {
return errors.Wrap(err, "transferring k8s binaries")
}
// Create image tarball
if err := createImageTarball(tarballFilename); err != nil {
return errors.Wrap(err, "create tarball")
}
return copyTarballToHost(tarballFilename)
}
func createImageTarball(tarballFilename string) error {
dirs := []string{
fmt.Sprintf("./lib/docker/%s", dockerStorageDriver),
"./lib/docker/image",
"./lib/minikube/binaries",
}
args := []string{"exec", profile, "sudo", "tar", "-I", "lz4", "-C", "/var", "-cvf", tarballFilename}
args = append(args, dirs...)
cmd := exec.Command("docker", args...)
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "tarball cmd: %s", cmd.Args)
}
return nil
}
func copyTarballToHost(tarballFilename string) error {
dest := filepath.Join("out/", tarballFilename)
cmd := exec.Command("docker", "cp", fmt.Sprintf("%s:/%s", profile, tarballFilename), dest)
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "cp cmd: %s", cmd.Args)
}
return nil
}
func deleteMinikube() error {
cmd := exec.Command(minikubePath, "delete", "-p", profile)
cmd.Stdout = os.Stdout
return cmd.Run()
}
/*
Copyright 2020 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"context"
"github.com/golang/glog"
"github.com/google/go-github/github"
)
// RecentK8sVersions returns the most recent k8s version, usually around 30
func RecentK8sVersions() ([]string, error) {
client := github.NewClient(nil)
k8s := "kubernetes"
list, _, err := client.Repositories.ListReleases(context.Background(), k8s, k8s, &github.ListOptions{})
if err != nil {
return nil, err
}
var releases []string
for _, r := range list {
releases = append(releases, r.GetTagName())
}
glog.Infof("Got releases: %v", releases)
return releases, nil
}
......@@ -20,20 +20,11 @@ import (
"bytes"
"flag"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"github.com/pkg/errors"
"k8s.io/minikube/pkg/drivers/kic"
"k8s.io/minikube/pkg/drivers/kic/oci"
"k8s.io/minikube/pkg/minikube/bootstrapper/bsutil"
"k8s.io/minikube/pkg/minikube/bootstrapper/images"
"k8s.io/minikube/pkg/minikube/command"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/driver"
"k8s.io/minikube/pkg/minikube/localpath"
"k8s.io/minikube/pkg/minikube/download"
"k8s.io/minikube/pkg/minikube/exit"
)
const (
......@@ -42,123 +33,55 @@ const (
)
var (
kubernetesVersion = ""
tarballFilename = ""
dockerStorageDriver = ""
preloadedTarballVersion = ""
containerRuntime = ""
dockerStorageDriver = "overlay2"
containerRuntimes = []string{"docker"}
k8sVersion string
k8sVersions []string
)
func init() {
flag.StringVar(&kubernetesVersion, "kubernetes-version", "", "desired kubernetes version, for example `v1.17.2`")
flag.StringVar(&dockerStorageDriver, "docker-storage-driver", "overlay2", "docker storage driver backend")
flag.StringVar(&preloadedTarballVersion, "preloaded-tarball-version", "", "preloaded tarball version")
flag.StringVar(&containerRuntime, "container-runtime", "docker", "container runtime")
flag.StringVar(&k8sVersion, "kubernetes-version", "", "desired kubernetes version, for example `v1.17.2`")
flag.Parse()
tarballFilename = fmt.Sprintf("preloaded-images-k8s-%s-%s-%s-%s.tar.lz4", preloadedTarballVersion, kubernetesVersion, containerRuntime, dockerStorageDriver)
if k8sVersion != "" {
k8sVersions = append(k8sVersions, k8sVersion)
}
}
func main() {
if err := verifyDockerStorage(); err != nil {
fmt.Println(err)
os.Exit(1)
exit.WithError("Docker storage type is incompatible: %v\n", err)
}
if err := executePreloadImages(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
func executePreloadImages() error {
defer func() {
if err := deleteMinikube(); err != nil {
fmt.Println(err)
if k8sVersions == nil {
var err error
k8sVersions, err = RecentK8sVersions()
if err != nil {
exit.WithError("Unable to get recent k8s versions: %v\n", err)
}
}()
driver := kic.NewDriver(kic.Config{
KubernetesVersion: kubernetesVersion,
ContainerRuntime: driver.Docker,
OCIBinary: oci.Docker,
MachineName: profile,
ImageDigest: kic.BaseImage,
StorePath: localpath.MiniPath(),
CPU: 2,
Memory: 4000,
APIServerPort: 8080,
})
baseDir := filepath.Dir(driver.GetSSHKeyPath())
defer os.Remove(baseDir)
if err := os.MkdirAll(baseDir, 0755); err != nil {
return errors.Wrap(err, "mkdir")
}
if err := driver.Create(); err != nil {
return errors.Wrap(err, "creating kic driver")
}
// Now, get images to pull
imgs, err := images.Kubeadm("", kubernetesVersion)
if err != nil {
return errors.Wrap(err, "kubeadm images")
}
for _, img := range append(imgs, kic.OverlayImage) {
cmd := exec.Command("docker", "exec", profile, "docker", "pull", img)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "downloading %s", img)
for _, kv := range k8sVersions {
for _, cr := range containerRuntimes {
tf := download.TarballName(kv)
if tarballExists(tf) {
fmt.Printf("A preloaded tarball for k8s version %s already exists, skipping generation.\n", kv)
continue
}
fmt.Printf("A preloaded tarball for k8s version %s doesn't exist, generating now...\n", kv)
if err := generateTarball(kv, tf); err != nil {
exit.WithError(fmt.Sprintf("generating tarball for k8s version %s with %s", kv, cr), err)
}
if err := uploadTarball(tf); err != nil {
exit.WithError(fmt.Sprintf("uploading tarball for k8s version %s with %s", kv, cr), err)
}
}
}
// Transfer in k8s binaries
kcfg := config.KubernetesConfig{
KubernetesVersion: kubernetesVersion,
}
runner := command.NewKICRunner(profile, driver.OCIBinary)
if err := bsutil.TransferBinaries(kcfg, runner); err != nil {
return errors.Wrap(err, "transferring k8s binaries")
}
// Create image tarball
if err := createImageTarball(); err != nil {
return errors.Wrap(err, "create tarball")
}
return copyTarballToHost()
}
func createImageTarball() error {
dirs := []string{
fmt.Sprintf("./lib/docker/%s", dockerStorageDriver),
"./lib/docker/image",
"./lib/minikube/binaries",
}
args := []string{"exec", profile, "sudo", "tar", "-I", "lz4", "-C", "/var", "-cvf", tarballFilename}
args = append(args, dirs...)
cmd := exec.Command("docker", args...)
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "tarball cmd: %s", cmd.Args)
}
return nil
}
func copyTarballToHost() error {
dest := filepath.Join("out/", tarballFilename)
cmd := exec.Command("docker", "cp", fmt.Sprintf("%s:/%s", profile, tarballFilename), dest)
cmd.Stdout = os.Stdout
if err := cmd.Run(); err != nil {
return errors.Wrapf(err, "cp cmd: %s", cmd.Args)
}
return nil
}
func deleteMinikube() error {
cmd := exec.Command(minikubePath, "delete", "-p", profile)
cmd.Stdout = os.Stdout
return cmd.Run()
func tarballExists(tarballFilename string) bool {
fmt.Println("Checking if tarball already exists...")
gcsPath := fmt.Sprintf("gs://%s/%s", download.PreloadBucket, tarballFilename)
cmd := exec.Command("gsutil", "stat", gcsPath)
return cmd.Run() == nil
}
func verifyDockerStorage() error {
......
/*
Copyright 2020 The Kubernetes Authors All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package main
import (
"fmt"
"os/exec"
"path"
"github.com/pkg/errors"
"k8s.io/minikube/pkg/minikube/download"
)
func uploadTarball(tarballFilename string) error {
// Upload tarball to GCS
hostPath := path.Join("out/", tarballFilename)
gcsDest := fmt.Sprintf("gs://%s", download.PreloadBucket)
cmd := exec.Command("gsutil", "cp", hostPath, gcsDest)
if output, err := cmd.Output(); err != nil {
return errors.Wrapf(err, "uploading %s to GCS bucket: %v\n%s", hostPath, err, string(output))
}
// Make tarball public to all users
gcsPath := fmt.Sprintf("%s/%s", gcsDest, tarballFilename)
cmd = exec.Command("gsutil", "acl", "ch", "-u", "AllUsers:R", gcsPath)
if output, err := cmd.Output(); err != nil {
return errors.Wrapf(err, "uploading %s to GCS bucket: %v\n%s", hostPath, err, string(output))
}
return nil
}
......@@ -112,6 +112,13 @@ var Addons = []*Addon{
set: SetBool,
callbacks: []setFn{enableOrDisableAddon},
},
{
name: "registry-aliases",
set: SetBool,
callbacks: []setFn{enableOrDisableAddon},
//TODO - add other settings
//TODO check if registry addon is enabled
},
{
name: "storage-provisioner",
set: SetBool,
......
......@@ -246,6 +246,38 @@ var Addons = map[string]*Addon{
"0640",
false),
}, false, "registry-creds"),
"registry-aliases": NewAddon([]*BinAsset{
MustBinAsset(
"deploy/addons/registry-aliases/registry-aliases-sa.tmpl",
vmpath.GuestAddonsDir,
"registry-aliases-sa.yaml",
"0640",
false),
MustBinAsset(
"deploy/addons/registry-aliases/registry-aliases-sa-crb.tmpl",
vmpath.GuestAddonsDir,
"registry-aliases-sa-crb.yaml",
"0640",
false),
MustBinAsset(
"deploy/addons/registry-aliases/registry-aliases-config.tmpl",
vmpath.GuestAddonsDir,
"registry-aliases-config.yaml",
"0640",
false),
MustBinAsset(
"deploy/addons/registry-aliases/node-etc-hosts-update.tmpl",
vmpath.GuestAddonsDir,
"node-etc-hosts-update.yaml",
"0640",
false),
MustBinAsset(
"deploy/addons/registry-aliases/patch-coredns-job.tmpl",
vmpath.GuestAddonsDir,
"patch-coredns-job.yaml",
"0640",
false),
}, false, "registry-aliases"),
"freshpod": NewAddon([]*BinAsset{
MustBinAsset(
"deploy/addons/freshpod/freshpod-rc.yaml.tmpl",
......
......@@ -81,7 +81,7 @@ Wants=crio.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.18.0-beta.2/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/var/run/crio/crio.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=/var/run/crio/crio.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
ExecStart=/var/lib/minikube/binaries/v1.18.0-rc.1/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=/var/run/crio/crio.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=/var/run/crio/crio.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
[Install]
`,
......@@ -107,7 +107,7 @@ Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.17.3/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
ExecStart=/var/lib/minikube/binaries/v1.18.0-rc.1/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
[Install]
`,
......@@ -140,7 +140,7 @@ Wants=containerd.service
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.17.3/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.200 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
ExecStart=/var/lib/minikube/binaries/v1.18.0-rc.1/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock --fail-swap-on=false --hostname-override=minikube --image-service-endpoint=unix:///run/containerd/containerd.sock --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.200 --pod-manifest-path=/etc/kubernetes/manifests --runtime-request-timeout=15m
[Install]
`,
......@@ -167,7 +167,7 @@ Wants=docker.socket
[Service]
ExecStart=
ExecStart=/var/lib/minikube/binaries/v1.17.3/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=docker --fail-swap-on=false --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-infra-container-image=docker-proxy-image.io/google_containers/pause:3.1 --pod-manifest-path=/etc/kubernetes/manifests
ExecStart=/var/lib/minikube/binaries/v1.18.0-rc.1/kubelet --authorization-mode=Webhook --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --cgroup-driver=cgroupfs --client-ca-file=/var/lib/minikube/certs/ca.crt --cluster-domain=cluster.local --config=/var/lib/kubelet/config.yaml --container-runtime=docker --fail-swap-on=false --hostname-override=minikube --kubeconfig=/etc/kubernetes/kubelet.conf --node-ip=192.168.1.100 --pod-infra-container-image=docker-proxy-image.io/google_containers/pause:3.2 --pod-manifest-path=/etc/kubernetes/manifests
[Install]
`,
......
......@@ -34,18 +34,28 @@ import (
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/client-go/kubernetes"
kconst "k8s.io/kubernetes/cmd/kubeadm/app/constants"
"k8s.io/minikube/pkg/minikube/bootstrapper"
"k8s.io/minikube/pkg/minikube/command"
"k8s.io/minikube/pkg/minikube/cruntime"
"k8s.io/minikube/pkg/minikube/logs"
)
// APIServerProcess waits for api server to be healthy returns error if it doesn't
func APIServerProcess(runner command.Runner, start time.Time, timeout time.Duration) error {
// WaitForAPIServerProcess waits for api server to be healthy returns error if it doesn't
func WaitForAPIServerProcess(r cruntime.Manager, bs bootstrapper.Bootstrapper, cr command.Runner, start time.Time, timeout time.Duration) error {
glog.Infof("waiting for apiserver process to appear ...")
minLogTime := kconst.APICallRetryInterval * 10
err := wait.PollImmediate(time.Millisecond*500, timeout, func() (bool, error) {
if time.Since(start) > timeout {
return false, fmt.Errorf("cluster wait timed out during process check")
}
if _, ierr := apiServerPID(runner); ierr != nil {
if time.Since(start) > minLogTime {
announceProblems(r, bs, cr)
time.Sleep(kconst.APICallRetryInterval * 5)
}
if _, ierr := apiServerPID(cr); ierr != nil {
return false, nil
}
return true, nil
......@@ -67,14 +77,21 @@ func apiServerPID(cr command.Runner) (int, error) {
return strconv.Atoi(s)
}
// SystemPods verifies essential pods for running kubernetes is running
func SystemPods(client *kubernetes.Clientset, start time.Time, timeout time.Duration) error {
// WaitForSystemPods verifies essential pods for running kurnetes is running
func WaitForSystemPods(r cruntime.Manager, bs bootstrapper.Bootstrapper, cr command.Runner, client *kubernetes.Clientset, start time.Time, timeout time.Duration) error {
glog.Info("waiting for kube-system pods to appear ...")
pStart := time.Now()
minLogTime := kconst.APICallRetryInterval * 10
podList := func() (bool, error) {
if time.Since(start) > timeout {
return false, fmt.Errorf("cluster wait timed out during pod check")
}
if time.Since(start) > minLogTime {
announceProblems(r, bs, cr)
time.Sleep(kconst.APICallRetryInterval * 5)
}
// Wait for any system pod, as waiting for apiserver may block until etcd
pods, err := client.CoreV1().Pods("kube-system").List(meta.ListOptions{})
if err != nil {
......@@ -94,15 +111,22 @@ func SystemPods(client *kubernetes.Clientset, start time.Time, timeout time.Dura
return nil
}
// APIServerIsRunning waits for api server status to be running
func APIServerIsRunning(start time.Time, ip string, port int, timeout time.Duration) error {
// WaitForHealthyAPIServer waits for api server status to be running
func WaitForHealthyAPIServer(r cruntime.Manager, bs bootstrapper.Bootstrapper, cr command.Runner, start time.Time, ip string, port int, timeout time.Duration) error {
glog.Infof("waiting for apiserver healthz status ...")
hStart := time.Now()
minLogTime := kconst.APICallRetryInterval * 10
healthz := func() (bool, error) {
if time.Since(start) > timeout {
return false, fmt.Errorf("cluster wait timed out during healthz check")
}
if time.Since(start) > minLogTime {
announceProblems(r, bs, cr)
time.Sleep(kconst.APICallRetryInterval * 5)
}
status, err := apiServerHealthz(net.ParseIP(ip), port)
if err != nil {
glog.Warningf("status: %v", err)
......@@ -121,6 +145,15 @@ func APIServerIsRunning(start time.Time, ip string, port int, timeout time.Durat
return nil
}
// announceProblems checks for problems, and slows polling down if any are found
func announceProblems(r cruntime.Manager, bs bootstrapper.Bootstrapper, cr command.Runner) {
problems := logs.FindProblems(r, bs, cr)
if len(problems) > 0 {
logs.OutputProblems(problems, 5)
time.Sleep(kconst.APICallRetryInterval * 15)
}
}
// APIServerStatus returns apiserver status in libmachine style state.State
func APIServerStatus(cr command.Runner, ip net.IP, port int) (state.State, error) {
glog.Infof("Checking apiserver status ...")
......@@ -175,6 +208,10 @@ func apiServerHealthz(ip net.IP, port int) (state.State, error) {
if err != nil {
return state.Stopped, nil
}
if resp.StatusCode == http.StatusUnauthorized {
glog.Errorf("%s returned code %d (unauthorized). Please ensure that your apiserver authorization settings make sense!", url, resp.StatusCode)
return state.Error, nil
}
if resp.StatusCode != http.StatusOK {
glog.Warningf("%s response: %v %+v", url, err, resp)
return state.Error, nil
......
......@@ -135,7 +135,7 @@ func dashboardFrontend(repo string) string {
repo = "kubernetesui"
}
// See 'kubernetes-dashboard' in deploy/addons/dashboard/dashboard-dp.yaml
return path.Join(repo, "dashboard:v2.0.0-beta8")
return path.Join(repo, "dashboard:v2.0.0-rc6")
}
// dashboardMetrics returns the image used for the dashboard metrics scraper
......
......@@ -25,7 +25,7 @@ import (
func TestAuxiliary(t *testing.T) {
want := []string{
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}
got := auxiliary("")
......@@ -37,7 +37,7 @@ func TestAuxiliary(t *testing.T) {
func TestAuxiliaryMirror(t *testing.T) {
want := []string{
"test.mirror/storage-provisioner:v1.8.1",
"test.mirror/dashboard:v2.0.0-beta8",
"test.mirror/dashboard:v2.0.0-rc6",
"test.mirror/metrics-scraper:v1.0.2",
}
got := auxiliary("test.mirror")
......
......@@ -38,7 +38,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd:3.4.3-0",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
{"v1.16.1", "mirror.k8s.io", []string{
......@@ -50,7 +50,7 @@ func TestKubeadmImages(t *testing.T) {
"mirror.k8s.io/etcd:3.3.15-0",
"mirror.k8s.io/pause:3.1",
"mirror.k8s.io/storage-provisioner:v1.8.1",
"mirror.k8s.io/dashboard:v2.0.0-beta8",
"mirror.k8s.io/dashboard:v2.0.0-rc6",
"mirror.k8s.io/metrics-scraper:v1.0.2",
}},
{"v1.15.0", "", []string{
......@@ -62,7 +62,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd:3.3.10",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
{"v1.14.0", "", []string{
......@@ -74,7 +74,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd:3.3.10",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
{"v1.13.0", "", []string{
......@@ -86,7 +86,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd:3.2.24",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
{"v1.12.0", "", []string{
......@@ -98,7 +98,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd:3.2.24",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
{"v1.11.10", "", []string{
......@@ -110,7 +110,7 @@ func TestKubeadmImages(t *testing.T) {
"k8s.gcr.io/etcd-amd64:3.2.18",
"k8s.gcr.io/pause:3.1",
"gcr.io/k8s-minikube/storage-provisioner:v1.8.1",
"kubernetesui/dashboard:v2.0.0-beta8",
"kubernetesui/dashboard:v2.0.0-rc6",
"kubernetesui/metrics-scraper:v1.0.2",
}},
}
......
......@@ -264,10 +264,15 @@ func (k *Bootstrapper) client(ip string, port int) (*kubernetes.Clientset, error
// WaitForNode blocks until the node appears to be healthy
func (k *Bootstrapper) WaitForNode(cfg config.ClusterConfig, n config.Node, timeout time.Duration) error {
start := time.Now()
out.T(out.Waiting, "Waiting for node {{.name}} to come online ...", out.V{"name": n.Name})
out.T(out.Waiting, "Waiting for cluster to come online ...")
cr, err := cruntime.New(cruntime.Config{Type: cfg.KubernetesConfig.ContainerRuntime, Runner: k.c})
if err != nil {
return err
}
if n.ControlPlane {
if err := kverify.APIServerProcess(k.c, start, timeout); err != nil {
if err := kverify.WaitForAPIServerProcess(cr, k, k.c, start, timeout); err != nil {
return err
}
}
......@@ -282,8 +287,9 @@ func (k *Bootstrapper) WaitForNode(cfg config.ClusterConfig, n config.Node, time
}
port = p
}
if n.ControlPlane {
if err := kverify.APIServerIsRunning(start, ip, port, timeout); err != nil {
if err := kverify.WaitForHealthyAPIServer(cr, k, k.c, start, ip, port, timeout); err != nil {
return err
}
}
......@@ -293,7 +299,7 @@ func (k *Bootstrapper) WaitForNode(cfg config.ClusterConfig, n config.Node, time
return errors.Wrap(err, "get k8s client")
}
if err := kverify.SystemPods(c, start, timeout); err != nil {
if err := kverify.WaitForSystemPods(cr, k, k.c, c, start, timeout); err != nil {
return errors.Wrap(err, "waiting for system pods")
}
return nil
......@@ -340,8 +346,13 @@ func (k *Bootstrapper) restartCluster(cfg config.ClusterConfig) error {
}
}
cr, err := cruntime.New(cruntime.Config{Type: cfg.KubernetesConfig.ContainerRuntime, Runner: k.c})
if err != nil {
return err
}
// We must ensure that the apiserver is healthy before proceeding
if err := kverify.APIServerProcess(k.c, time.Now(), kconst.DefaultControlPlaneTimeout); err != nil {
if err := kverify.WaitForAPIServerProcess(cr, k, k.c, time.Now(), kconst.DefaultControlPlaneTimeout); err != nil {
return errors.Wrap(err, "apiserver healthz")
}
......@@ -363,7 +374,7 @@ func (k *Bootstrapper) restartCluster(cfg config.ClusterConfig) error {
return errors.Wrap(err, "getting k8s client")
}
if err := kverify.SystemPods(client, time.Now(), kconst.DefaultControlPlaneTimeout); err != nil {
if err := kverify.WaitForSystemPods(cr, k, k.c, client, time.Now(), kconst.DefaultControlPlaneTimeout); err != nil {
return errors.Wrap(err, "system pods")
}
......@@ -582,7 +593,7 @@ func (k *Bootstrapper) applyNodeLabels(cfg config.ClusterConfig) error {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// example:
// sudo /var/lib/minikube/binaries/v1.17.3/kubectl label nodes minikube.k8s.io/version=v1.7.3 minikube.k8s.io/commit=aa91f39ffbcf27dcbb93c4ff3f457c54e585cf4a-dirty minikube.k8s.io/name=p1 minikube.k8s.io/updated_at=2020_02_20T12_05_35_0700 --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
// sudo /var/lib/minikube/binaries/<version>/kubectl label nodes minikube.k8s.io/version=<version> minikube.k8s.io/commit=aa91f39ffbcf27dcbb93c4ff3f457c54e585cf4a-dirty minikube.k8s.io/name=p1 minikube.k8s.io/updated_at=2020_02_20T12_05_35_0700 --all --overwrite --kubeconfig=/var/lib/minikube/kubeconfig
cmd := exec.CommandContext(ctx, "sudo",
path.Join(vmpath.GuestPersistentDir, "binaries", cfg.KubernetesConfig.KubernetesVersion, "kubectl"),
"label", "nodes", verLbl, commitLbl, nameLbl, createdAtLbl, "--all", "--overwrite",
......
......@@ -26,9 +26,9 @@ import (
const (
// DefaultKubernetesVersion is the default kubernetes version
DefaultKubernetesVersion = "v1.17.3"
DefaultKubernetesVersion = "v1.18.0-rc.1"
// NewestKubernetesVersion is the newest Kubernetes version to test against
NewestKubernetesVersion = "v1.18.0-beta.2"
NewestKubernetesVersion = "v1.18.0-rc.1"
// OldestKubernetesVersion is the oldest Kubernetes version to test against
OldestKubernetesVersion = "v1.11.10"
// DefaultClusterName is the default nane for the k8s cluster
......
......@@ -37,19 +37,21 @@ import (
const (
// PreloadVersion is the current version of the preloaded tarball
PreloadVersion = "v1"
//
// NOTE: You may need to bump this version up when upgrading auxiliary docker images
PreloadVersion = "v2"
// PreloadBucket is the name of the GCS bucket where preloaded volume tarballs exist
PreloadBucket = "minikube-preloaded-volume-tarballs"
)
// returns name of the tarball
func tarballName(k8sVersion string) string {
return fmt.Sprintf("preloaded-images-k8s-%s-%s-docker-overlay2.tar.lz4", PreloadVersion, k8sVersion)
// TarballName returns name of the tarball
func TarballName(k8sVersion string) string {
return fmt.Sprintf("preloaded-images-k8s-%s-%s-docker-overlay2-amd64.tar.lz4", PreloadVersion, k8sVersion)
}
// returns the name of the checksum file
func checksumName(k8sVersion string) string {
return fmt.Sprintf("%s.checksum", tarballName(k8sVersion))
return fmt.Sprintf("%s.checksum", TarballName(k8sVersion))
}
// returns target dir for all cached items related to preloading
......@@ -64,12 +66,12 @@ func PreloadChecksumPath(k8sVersion string) string {
// TarballPath returns the path to the preloaded tarball
func TarballPath(k8sVersion string) string {
return path.Join(targetDir(), tarballName(k8sVersion))
return path.Join(targetDir(), TarballName(k8sVersion))
}
// remoteTarballURL returns the URL for the remote tarball in GCS
func remoteTarballURL(k8sVersion string) string {
return fmt.Sprintf("https://storage.googleapis.com/%s/%s", PreloadBucket, tarballName(k8sVersion))
return fmt.Sprintf("https://storage.googleapis.com/%s/%s", PreloadBucket, TarballName(k8sVersion))
}
// PreloadExists returns true if there is a preloaded tarball that can be used
......@@ -147,13 +149,13 @@ func Preload(k8sVersion, containerRuntime string) error {
}
func saveChecksumFile(k8sVersion string) error {
glog.Infof("saving checksum for %s ...", tarballName(k8sVersion))
glog.Infof("saving checksum for %s ...", TarballName(k8sVersion))
ctx := context.Background()
client, err := storage.NewClient(ctx, option.WithoutAuthentication())
if err != nil {
return errors.Wrap(err, "getting storage client")
}
attrs, err := client.Bucket(PreloadBucket).Object(tarballName(k8sVersion)).Attrs(ctx)
attrs, err := client.Bucket(PreloadBucket).Object(TarballName(k8sVersion)).Attrs(ctx)
if err != nil {
return errors.Wrap(err, "getting storage object")
}
......
......@@ -35,8 +35,33 @@ import (
"k8s.io/minikube/pkg/minikube/out"
)
// rootCauseRe is a regular expression that matches known failure root causes
var rootCauseRe = regexp.MustCompile(`^error: |eviction manager: pods.* evicted|unknown flag: --|forbidden.*no providers available|eviction manager:.*evicted|tls: bad certificate|kubelet.*no API client|kubelet.*No api server|STDIN.*127.0.0.1:8080|failed to create listener|address already in use|unable to evict any pods|eviction manager: unexpected error`)
// rootCauses are regular expressions that match known failures
var rootCauses = []string{
`^error: `,
`eviction manager: pods.* evicted`,
`unknown flag: --`,
`forbidden.*no providers available`,
`eviction manager:.*evicted`,
`tls: bad certificate`,
`kubelet.*no API client`,
`kubelet.*No api server`,
`STDIN.*127.0.0.1:8080`,
`failed to create listener`,
`address already in use`,
`unable to evict any pods`,
`eviction manager: unexpected error`,
`Resetting AnonymousAuth to false`,
`CrashLoopBackOff`,
`Unable to register node.*forbidden`,
`Failed to initialize CSINodeInfo.*forbidden`,
`Failed to admit pod`,
`failed to "StartContainer"`,
`kubelet.*forbidden.*cannot \w+ resource`,
`leases.*forbidden.*cannot \w+ resource`,
}
// rootCauseRe combines rootCauses into a single regex
var rootCauseRe = regexp.MustCompile(strings.Join(rootCauses, "|"))
// ignoreCauseRe is a regular expression that matches spurious errors to not surface
var ignoreCauseRe = regexp.MustCompile("error: no objects passed to apply")
......@@ -44,6 +69,7 @@ var ignoreCauseRe = regexp.MustCompile("error: no objects passed to apply")
// importantPods are a list of pods to retrieve logs for, in addition to the bootstrapper logs.
var importantPods = []string{
"kube-apiserver",
"etcd",
"coredns",
"kube-scheduler",
"kube-proxy",
......
......@@ -36,6 +36,19 @@ func TestIsProblem(t *testing.T) {
{"no-objects-passed-to-apply #4010", false, "error: no objects passed to apply"},
{"bad-certificate #4251", true, "log.go:172] http: TLS handshake error from 127.0.0.1:49200: remote error: tls: bad certificate"},
{"ephemeral-eviction #5355", true, " eviction_manager.go:419] eviction manager: unexpected error when attempting to reduce ephemeral-storage pressure: wanted to free 9223372036854775807 bytes, but freed 0 bytes space with errors in image deletion"},
{"anonymous-auth", true, "AnonymousAuth is not allowed with the AlwaysAllow authorizer. Resetting AnonymousAuth to false. You should use a different authorizer"},
{"disk-pressure #7073", true, "eviction_manager.go:159] Failed to admit pod kindnet-jpzzf_kube-system(b63b1ee0-0fc6-428f-8e67-e357464f579c) - node has conditions: [DiskPressure]"},
{"csi timeout", true, `Failed to initialize CSINodeInfo: error updating CSINode annotation: timed out waiting for the condition; caused by: csinodes.storage.k8s.io "m01" is forbidden: User "system:node:m01" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope`},
{"node registration permissions", true, `Unable to register node "m01" with API server: nodes is forbidden: User "system:node:m01" cannot create resource "nodes" in API group "" at the cluster scope`},
{"regular kubelet refused", false, `kubelet_node_status.go:92] Unable to register node "m01" with API server: Post https://localhost:8443/api/v1/nodes: dial tcp 127.0.0.1:8443: connect: connection refused`},
{"regular csi refused", false, `Failed to initialize CSINodeInfo: error updating CSINode annotation: timed out waiting for the condition; caused by: Get https://localhost:8443/apis/storage.k8s.io/v1/csinodes/m01: dial tcp 127.0.0.1:8443: connect: connection refused`},
{"apiserver crashloop", true, `pod_workers.go:191] Error syncing pod 9f8ee739bd14e8733f807eb2be99768f ("kube-apiserver-m01_kube-system(9f8ee739bd14e8733f807eb2be99768f)"), skipping: failed to "StartContainer" for "kube-apiserver" with CrashLoopBackOff: "back-off 10s restarting failed container=kube-apiserver pod=kube-apiserver-m01_kube-system(9f8ee739bd14e8733f807eb2be99768f)`},
{"kubelet node timeout", false, `failed to ensure node lease exists, will retry in 6.4s, error: Get https://localhost:8443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/m01?timeout=10s: dial tcp 127.0.0.1:8443: connect: connection refused`},
{"rbac misconfiguration", true, `leases.coordination.k8s.io "m01" is forbidden: User "system:node:m01" cannot get resource "leases" in API group "coordination.k8s.io" in the namespace "kube-node-lease"`},
{"regular controller init", false, `error retrieving resource lock kube-system/kube-controller-manager: endpoints "kube-controller-manager" is forbidden: User "system:kube-controller-manager" cannot get resource "endpoints" in API group "" in the namespace "kube-system"`},
{"regular scheduler services init", false, ` k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Service: services is forbidden: User "system:kube-scheduler" cannot list resource "services" in API group "" at the cluster scope`},
{"regular scheduler nodes init", false, `k8s.io/client-go/informers/factory.go:135: Failed to list *v1.Node: nodes is forbidden: User "system:kube-scheduler" cannot list resource "nodes" in API group "" at the cluster scope`},
{"kubelet rbac fail", true, `k8s.io/kubernetes/pkg/kubelet/kubelet.go:526: Failed to list *v1.Node: nodes "m01" is forbidden: User "system:node:m01" cannot list resource "nodes" in API group "" at the cluster scope`},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
......
......@@ -20,6 +20,7 @@ import (
"context"
"fmt"
"os/exec"
"runtime"
"time"
"github.com/docker/machine/libmachine/drivers"
......@@ -32,12 +33,21 @@ import (
)
func init() {
priority := registry.Default
// Staged rollout for preferred:
// - Linux
// - Windows (once "service" command works)
// - macOS
if runtime.GOOS == "linux" {
priority = registry.Preferred
}
if err := registry.Register(registry.DriverDef{
Name: driver.Docker,
Config: configure,
Init: func() drivers.Driver { return kic.NewDriver(kic.Config{OCIBinary: oci.Docker}) },
Status: status,
Priority: registry.Fallback,
Priority: priority,
}); err != nil {
panic(fmt.Sprintf("register failed: %v", err))
}
......
......@@ -28,3 +28,4 @@ As an experimental driver, not all commands are supported on all platforms. Nota
## Troubleshooting
* Run `minikube start --alsologtostderr -v=1` to debug crashes
* If your docker is too slow on mac os try [Improving docker performance](https://docs.docker.com/docker-for-mac/osxfs-caching/)
......@@ -32,9 +32,6 @@ import (
"testing"
"time"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/daemon"
"k8s.io/minikube/pkg/drivers/kic"
"k8s.io/minikube/pkg/minikube/bootstrapper/images"
"k8s.io/minikube/pkg/minikube/config"
"k8s.io/minikube/pkg/minikube/constants"
......@@ -74,6 +71,14 @@ func TestDownloadOnly(t *testing.T) {
t.Errorf("%s failed: %v", args, err)
}
if download.PreloadExists(v, "docker") {
// Just make sure the tarball path exists
if _, err := os.Stat(download.TarballPath(v)); err != nil {
t.Errorf("preloaded tarball path doesn't exist: %v", err)
}
return
}
imgs, err := images.Kubeadm("", v)
if err != nil {
t.Errorf("kubeadm images: %v %+v", v, err)
......@@ -197,15 +202,6 @@ func TestDownloadOnlyDocker(t *testing.T) {
if string(remoteChecksum) != string(checksum[:]) {
t.Errorf("checksum of %s does not match remote checksum (%s != %s)", tarball, string(remoteChecksum), string(checksum[:]))
}
// Make sure this image exists in the docker daemon
ref, err := name.ParseReference(kic.BaseImage)
if err != nil {
t.Errorf("parsing reference failed: %v", err)
}
if _, err := daemon.Image(ref); err != nil {
t.Errorf("expected image does not exist in local daemon: %v", err)
}
}
func runningDockerDriver(startArgs []string) bool {
......
......@@ -80,7 +80,7 @@
"Deletes a local kubernetes cluster. This command deletes the VM, and removes all associated files.": "Supprime le cluster Kubernetes local. Cette commande supprime la VM ainsi que tous les fichiers associés.",
"Deletes a node from a cluster.": "",
"Deleting \"{{.profile_name}}\" in {{.driver_name}} ...": "Suppression de \"{{.profile_name}}\" dans {{.driver_name}}...",
"Deleting node {{.name}} from cluster {{.cluster}}": "",
"Deleting node {{.name}} from cluster {{.cluster}}": "Suppression de noeuds {{.name}} de cluster {{.cluster}}",
"Disable checking for the availability of hardware virtualization before the vm is started (virtualbox driver only)": "Désactive la vérification de la disponibilité de la virtualisation du matériel avant le démarrage de la VM (pilote virtualbox uniquement).",
"Disable dynamic memory in your VM manager, or pass in a larger --memory value": "",
"Disables the addon w/ADDON_NAME within minikube (example: minikube addons disable dashboard). For a list of available addons use: minikube addons list": "",
......@@ -654,4 +654,4 @@
"{{.prefix}}minikube {{.version}} on {{.platform}}": "{{.prefix}}minikube {{.version}} sur {{.platform}}",
"{{.type}} is not yet a supported filesystem. We will try anyways!": "",
"{{.url}} is not accessible: {{.error}}": ""
}
\ No newline at end of file
}
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册