Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

[ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content#7180

Merged
RongtongJin merged 2 commits intoapache:developapache/rocketmq:developfrom
Ziy1-Tan:kv_jsonZiy1-Tan/rocketmq:kv_jsonCopy head branch name to clipboard
Aug 23, 2023
Merged

[ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content#7180
RongtongJin merged 2 commits intoapache:developapache/rocketmq:developfrom
Ziy1-Tan:kv_jsonZiy1-Tan/rocketmq:kv_jsonCopy head branch name to clipboard

Conversation

@Ziy1-Tan
Copy link
Copy Markdown
Contributor

@Ziy1-Tan Ziy1-Tan commented Aug 14, 2023

Which Issue(s) This PR Fixes

Fixes #7142

Brief Description

  • case get topic config
    mqadmin RocksDBConfigToJson -p /home/xxx/store/config/topics -t topics
	"topicConfigTable":{
		"SCHEDULE_TOPIC_XXXX":{
			"readQueueNums":18,
			"perm":6,
			"writeQueueNums":18,
			"topicFilterType":"SINGLE_TAG",
			"topicName":"SCHEDULE_TOPIC_XXXX",
			"attributes":{},
			"order":false,
			"topicSysFlag":0
		},
		"RMQ_SYS_TRANS_HALF_TOPIC":{
			"readQueueNums":1,
			"perm":6,
			"writeQueueNums":1,
			"topicFilterType":"SINGLE_TAG",
			"topicName":"RMQ_SYS_TRANS_HALF_TOPIC",
			"attributes":{},
			"order":false,
			"topicSysFlag":0
		},
// ...
}
  • case get subscription config
    mqadmin RocksDBConfigToJson -p /home/xxx/store/config/subscriptionGroups -t subscriptionGroups
{
	"subscriptionGroupTable":{
		"CID_ONSAPI_OWNER":{
			"retryMaxTimes":16,
			"whichBrokerWhenConsumeSlowly":1,
			"brokerId":0,
			"groupRetryPolicy":{
				"type":"CUSTOMIZED"
			},
			"consumeFromMinEnable":true,
			"consumeTimeoutMinute":15,
			"groupName":"CID_ONSAPI_OWNER",
			"groupSysFlag":0,
			"consumeEnable":true,
			"consumeBroadcastEnable":true,
			"retryQueueNums":1,
			"notifyConsumerIdsChangedEnable":true,
			"consumeMessageOrderly":false,
			"attributes":{}
		},
//...
}

How Did You Test This Change?

TBD

  • allow append to file

@RongtongJin RongtongJin changed the title feat: add command kvConfigToJson to inspect rocksdb content [ISSUE #7142] Add command kvConfigToJson to inspect rocksdb content Aug 14, 2023
import java.util.HashMap;
import java.util.Map;

public class KvConfigToJsonCommand implements SubCommand {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about renaming to RocksDBConfigToJsonCommand?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good! Let me take a look.

Signed-off-by: Ziy1-Tan <ajb4596984460@gmail.com>
@Ziy1-Tan Ziy1-Tan changed the title [ISSUE #7142] Add command kvConfigToJson to inspect rocksdb content [ISSUE #7142] Add command RocksDBConfigToJson to inspect rocksdb content Aug 14, 2023
@Override
public void execute(CommandLine commandLine, Options options, RPCHook rpcHook) throws SubCommandException {
String path = commandLine.getOptionValue("path").trim();
if (StringUtils.isEmpty(path) || !new File(path).exists()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support default path configuration?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metadata is stored in different paths, it may be more appropriate to specify the path.

}
topicsJsonConfig.put("topicConfigTable", (JSONObject) JSONObject.toJSON(topicConfigTable));
final String topicsJsonStr = JSONObject.toJSONString(topicsJsonConfig, true);
System.out.print(topicsJsonStr + "\n");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The topic size is too large, causing Out Of Memory (OOM) errors.

You could use the stream approach to write to the file, instead of reusing the previous code.

Copy link
Copy Markdown
Contributor Author

@Ziy1-Tan Ziy1-Tan Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about mqadmin rocksDBConfig -p /home/xxx/store/config/topics -t topics > topics.json?


@Override
public String commandName() {
return "RocksDBConfigToJson";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rocksDBConfigToJson, lowercase the first letter like any other command.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 15, 2023

Codecov Report

❌ Patch coverage is 24.00000% with 38 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.48%. Comparing base (f534501) to head (4366490).
⚠️ Report is 663 commits behind head on develop.

Files with missing lines Patch % Lines
...s/command/metadata/RocksDBConfigToJsonCommand.java 24.48% 36 Missing and 1 partial ⚠️
.../apache/rocketmq/tools/command/MQAdminStartup.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #7180      +/-   ##
=============================================
- Coverage      42.52%   42.48%   -0.05%     
- Complexity      9357     9363       +6     
=============================================
  Files           1149     1150       +1     
  Lines          82043    82093      +50     
  Branches       10701    10706       +5     
=============================================
- Hits           34889    34877      -12     
- Misses         42792    42850      +58     
- Partials        4362     4366       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@RongtongJin RongtongJin merged commit 744167b into apache:develop Aug 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add admin tool to inspect RocksDB contents

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.