Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 35956e9

Browse filesBrowse files
authored
Merge pull request #127667 from zylxjtu/dev
Add log file rotation
2 parents d96cbb1 + 0944665 commit 35956e9
Copy full SHA for 35956e9

File tree

Expand file treeCollapse file tree

6 files changed

+780
-9
lines changed
Filter options
Expand file treeCollapse file tree

6 files changed

+780
-9
lines changed

‎staging/src/k8s.io/component-base/logs/kube-log-runner/README.md

Copy file name to clipboardExpand all lines: staging/src/k8s.io/component-base/logs/kube-log-runner/README.md
+71Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,66 @@ Why do we need this?
1515
- Nowadays, the `--log-file` parameter is deprecated for Kubernetes components
1616
and should not be used anymore. `kube-log-runner` is a direct replacement.
1717

18+
## Flags
19+
20+
- -flush-interval
21+
The `-flush-interval` flag is a duration flag that specifies how frequently the log
22+
file content is flushed to disk. The default value is 0, meaning no periodic flushing.
23+
If set to a non-zero value, the log file is flushed at the specified interval.
24+
25+
Type: Duration
26+
27+
Default: 0 (flushing disabled)
28+
29+
Usage: When set to non-zero, the log file is flushed every specified interval.
30+
While this may not be necessary on Linux, on Windows, it ensures that recent log
31+
entries are written to disk in near real-time, which is particularly useful for
32+
users who need to monitor logs as they are generated without delay.
33+
34+
- -log-file-size
35+
The `-log-file-size` flag is an optional string flag that sets a size limit for
36+
the log file, triggering automatic log rotation when the specified size is reached.
37+
This is especially useful in production environments where the log file may become
38+
too large to view effectively.
39+
Beware that rotation can happen at arbitrary points in the byte stream emitted by the command.
40+
This can lead to splitting a log entry into two parts, with the first part in one file
41+
and the second part in the next file.
42+
43+
Type: String (expects a value in Resource.Quantity format, such as 10M or 500K)
44+
45+
Default: "0" (disabled, no automatic rotation of log files)
46+
47+
Usage: When set to a positive value, the log file will rotate upon reaching the specified
48+
size limit. The current log file’s contents will be saved to a backup file, and a new log
49+
file will be created at the path specified by the `-log-file` flag, ready for future log entries.
50+
51+
Backup File Naming Convention:
52+
`<original-file-name>-<timestamp><file-extension>`.
53+
* `<original-file-name>`: The name of the original log file, without the file extension.
54+
* `<timestamp>`: A timestamp is added to each backup file’s name to uniquely identify it
55+
based on the time it was created. The timestamp follows the format "20060102-150405".
56+
For example, a backup created on June 2, 2006, at 3:04:05 PM would include this timestamp.
57+
* `<file-extension>`: The original file’s extension (e.g., .log) remains unchanged.
58+
This naming convention ensures easy organization and retrieval of rotated log files based on their creation time.
59+
60+
- -log-file-age
61+
The `-log-file-age` flag is an optional time duration setting that defines how long
62+
old backup log files are retained. This flag is used alongside log rotation (enabled
63+
by setting a positive value for -log-file-size) to help manage storage by removing
64+
outdated backup logs.
65+
66+
Type: Duration
67+
68+
Default: 0 (disabled, no automatic deletion of backup files)
69+
70+
Usage: When -log-file-age is set to a positive duration (e.g., 24h for 24 hours)
71+
and log rotation is enabled, backup log files will be automatically deleted if
72+
the time when they were created (as encoded in the file name) is older than the
73+
specified duration from the current time.
74+
75+
This ensures that only recent backup logs are kept, preventing accumulation of old logs
76+
and reducing storage usage.
77+
1878
For example instead of running kube-apiserver like this:
1979
```bash
2080
"/bin/sh",
@@ -49,6 +109,17 @@ kube-log-runner -log-file=/tmp/log echo "hello world"
49109
# Copy into log file and print to stdout (same as 2>&1 | tee -a /tmp/log).
50110
kube-log-runner -log-file=/tmp/log -also-stdout echo "hello world"
51111
112+
# Copy into log file and print to stdout (same as 2>&1 | tee -a /tmp/log),
113+
# will flush the logging file in 5s,
114+
# rotate the log file when its size exceedes 10 MB
115+
kube-log-runner -flush-interval=5s -log-file=/tmp/log -log-file-size=10M -also-stdout echo "hello world"
116+
117+
# Copy into log file and print to stdout (same as 2>&1 | tee -a /tmp/log),
118+
# will flush the logging file in 10s,
119+
# rotate the log file when its size exceedes 10 MB,
120+
# and clean up old rotated log files when their age are older than 168h (7 days)
121+
kube-log-runner -flush-interval=10s -log-file=/tmp/log -log-file-size=10M -log-file-age=168h -also-stdout echo "hello world"
122+
52123
# Redirect only stdout into log file (same as 1>>/tmp/log).
53124
kube-log-runner -log-file=/tmp/log -redirect-stderr=false echo "hello world"
54125
```
+205Lines changed: 205 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,205 @@
1+
/*
2+
Copyright 2025 The Kubernetes Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package logrotation
18+
19+
import (
20+
"io"
21+
"os"
22+
"path/filepath"
23+
"strings"
24+
"sync"
25+
"time"
26+
)
27+
28+
const timeLayout string = "20060102-150405"
29+
30+
type rotationFile struct {
31+
// required, the max size of the log file in bytes, 0 means no rotation
32+
maxSize int64
33+
// required, the max age of the log file, 0 means no cleanup
34+
maxAge time.Duration
35+
filePath string
36+
mut sync.Mutex
37+
file *os.File
38+
currentSize int64
39+
lasSyncTime time.Time
40+
flushInterval time.Duration
41+
}
42+
43+
func Open(filePath string, flushInterval time.Duration, maxSize int64, maxAge time.Duration) (io.WriteCloser, error) {
44+
w := &rotationFile{
45+
filePath: filePath,
46+
maxSize: maxSize,
47+
maxAge: maxAge,
48+
flushInterval: flushInterval,
49+
}
50+
51+
logFile, err := os.OpenFile(w.filePath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
52+
if err != nil {
53+
return nil, err
54+
}
55+
56+
w.file = logFile
57+
58+
w.lasSyncTime = time.Now()
59+
60+
if w.maxSize > 0 {
61+
info, err := os.Stat(w.filePath)
62+
if err != nil {
63+
return nil, err
64+
}
65+
w.currentSize = info.Size()
66+
}
67+
68+
return w, nil
69+
}
70+
71+
// Write implements the io.Writer interface.
72+
func (w *rotationFile) Write(p []byte) (n int, err error) {
73+
w.mut.Lock()
74+
defer w.mut.Unlock()
75+
76+
n, err = w.file.Write(p)
77+
if err != nil {
78+
return 0, err
79+
}
80+
81+
if w.flushInterval > 0 && time.Since(w.lasSyncTime) >= w.flushInterval {
82+
err = w.file.Sync()
83+
if err != nil {
84+
return 0, err
85+
}
86+
w.lasSyncTime = time.Now()
87+
}
88+
89+
if w.maxSize > 0 {
90+
w.currentSize += int64(len(p))
91+
92+
// if file size over maxsize rotate the log file
93+
if w.currentSize >= w.maxSize {
94+
err = w.rotate()
95+
if err != nil {
96+
return 0, err
97+
}
98+
}
99+
}
100+
101+
return n, nil
102+
}
103+
104+
func (w *rotationFile) rotate() error {
105+
// Get the file extension
106+
ext := filepath.Ext(w.filePath)
107+
108+
// Remove the extension from the filename
109+
pathWithoutExt := strings.TrimSuffix(w.filePath, ext)
110+
111+
rotateFilePath := pathWithoutExt + "-" + time.Now().Format(timeLayout) + ext
112+
113+
if w.filePath == rotateFilePath {
114+
return nil
115+
}
116+
117+
err := w.file.Close()
118+
if err != nil {
119+
return err
120+
}
121+
122+
err = os.Rename(w.filePath, rotateFilePath)
123+
if err != nil {
124+
return err
125+
}
126+
127+
w.file, err = os.OpenFile(w.filePath, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
128+
if err != nil {
129+
return err
130+
}
131+
132+
w.currentSize = 0
133+
134+
if w.maxAge > 0 {
135+
go func() {
136+
err = w.clean(pathWithoutExt, ext)
137+
}()
138+
}
139+
140+
return nil
141+
}
142+
143+
// Clean up the old log files in the format of
144+
// <basename>-<timestamp><ext>.
145+
// This should be safe enough to avoid false deletion.
146+
// This will work for multiple restarts of the same program.
147+
func (w *rotationFile) clean(pathWithoutExt string, ext string) error {
148+
ageTime := time.Now().Add(-w.maxAge)
149+
150+
directory := filepath.Dir(pathWithoutExt)
151+
basename := filepath.Base(pathWithoutExt) + "-"
152+
153+
dir, err := os.ReadDir(directory)
154+
if err != nil {
155+
return err
156+
}
157+
158+
err = nil
159+
for _, v := range dir {
160+
if strings.HasPrefix(v.Name(), basename) && strings.HasSuffix(v.Name(), ext) {
161+
// Remove the prefix and suffix
162+
trimmed := strings.TrimPrefix(v.Name(), basename)
163+
trimmed = strings.TrimSuffix(trimmed, ext)
164+
165+
_, err = time.Parse(timeLayout, trimmed)
166+
if err == nil {
167+
info, errInfo := v.Info()
168+
if errInfo != nil {
169+
err = errInfo
170+
// Ignore the error while continue with the next clenup
171+
continue
172+
}
173+
174+
if ageTime.After(info.ModTime()) {
175+
err = os.Remove(filepath.Join(directory, v.Name()))
176+
if err != nil {
177+
// Ignore the error while continue with the next clenup
178+
continue
179+
}
180+
}
181+
}
182+
183+
}
184+
}
185+
186+
return err
187+
}
188+
189+
func (w *rotationFile) Close() error {
190+
w.mut.Lock()
191+
defer w.mut.Unlock()
192+
193+
// Explicitly call file.Sync() to ensure data is written to disk
194+
err := w.file.Sync()
195+
if err != nil {
196+
return err
197+
}
198+
199+
err = w.file.Close()
200+
if err != nil {
201+
return err
202+
}
203+
204+
return nil
205+
}

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.