report dirty metadata in meta table by Mengqi777 · Pull Request #81 · apache/hbase-operator-tools

Mengqi777 · 2021-02-09T02:51:24Z

Our cluster has experienced such problems： a table has been deleted but meta table still storage metadata about this table, thoese metadata show in hbck ui until I delete them.

Apache-HBase · 2021-02-09T02:58:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-0 ⚠️	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ master Compile Tests _
+1 💚	mvninstall	0m 48s	master passed
+1 💚	compile	0m 9s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 7s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 10s	the patch passed
+1 💚	compile	0m 9s	the patch passed
+1 💚	javac	0m 9s	the patch passed
-1 ❌	checkstyle	0m 4s	hbase-hbck2: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 5s	the patch passed
		_ Other Tests _
+1 💚	unit	3m 43s	hbase-hbck2 in the patch passed.
+1 💚	asflicense	0m 5s	The patch does not generate ASF License warnings.
		6m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/1/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#81
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 036fa5d67579 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `2958eeb`
Default Java	Oracle Corporation-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/1/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-hbck2.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/1/testReport/
Max. process+thread count	1030 (vs. ulimit of 5000)
modules	C: hbase-hbck2 U: hbase-hbck2
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/1/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil

Thanks for submitting this PR, @Mengqi777 ! I have a few remarks made on part of the code. Additional to those, please add UTs for each of the added methods.

Also, we need to add description for this new command at hbck2 README.md.

wchevreuil · 2021-08-16T10:30:26Z

Let's explain properly what "dirty" means in this context.

wchevreuil · 2021-08-16T10:35:54Z

Do we really need to iterate through each cell? It seems you only care about the row key, so no need to go through every single cell of each row.

wchevreuil · 2021-08-16T10:46:56Z

Let's give this command a more intuitive name. What's dirtyMetadata?

wchevreuil · 2021-08-16T10:48:33Z

Can we describe here in more details an example scenario where this command would be useful? See, for example, the explanation for reportMissingRegionsInMeta.

Apache-HBase · 2021-08-19T14:43:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 2s	#81 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#81
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/2/console
versions	git=2.17.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…tools

Apache-HBase · 2021-08-20T00:52:31Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	1m 4s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 7s	master passed
-0 ⚠️	patch	0m 18s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 11s	the patch passed
+1 💚	compile	0m 9s	the patch passed
+1 💚	javac	0m 9s	the patch passed
-1 ❌	checkstyle	0m 5s	hbase-hbck2: The patch generated 41 new + 0 unchanged - 0 fixed = 41 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 5s	the patch passed
		_ Other Tests _
-1 ❌	unit	3m 18s	hbase-hbck2 in the patch failed.
+1 💚	asflicense	0m 6s	The patch does not generate ASF License warnings.
		6m 12s

Reason	Tests
Failed junit tests	hbase.TestHBCK2

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/3/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#81
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux b501e887666b 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `90e0f45`
Default Java	Oracle Corporation-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/3/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-hbck2.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/3/artifact/yetus-precommit-check/output/patch-unit-hbase-hbck2.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/3/testReport/
Max. process+thread count	1204 (vs. ulimit of 5000)
modules	C: hbase-hbck2 U: hbase-hbck2
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/3/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-08-20T00:58:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	0m 51s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 7s	master passed
-0 ⚠️	patch	0m 18s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 11s	the patch passed
+1 💚	compile	0m 10s	the patch passed
+1 💚	javac	0m 10s	the patch passed
-1 ❌	checkstyle	0m 4s	hbase-hbck2: The patch generated 41 new + 0 unchanged - 0 fixed = 41 total (was 0)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 5s	the patch passed
		_ Other Tests _
-1 ❌	unit	3m 16s	hbase-hbck2 in the patch failed.
+1 💚	asflicense	0m 6s	The patch does not generate ASF License warnings.
		5m 53s

Reason	Tests
Failed junit tests	hbase.TestHBCK2

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/4/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#81
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux aa7e44d56c6f 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `90e0f45`
Default Java	Oracle Corporation-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/4/artifact/yetus-precommit-check/output/diff-checkstyle-hbase-hbck2.txt
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/4/artifact/yetus-precommit-check/output/patch-unit-hbase-hbck2.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/4/testReport/
Max. process+thread count	1198 (vs. ulimit of 5000)
modules	C: hbase-hbck2 U: hbase-hbck2
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/4/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-08-20T01:52:25Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 0s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	1m 2s	master passed
+1 💚	compile	0m 12s	master passed
+1 💚	checkstyle	0m 7s	master passed
+1 💚	javadoc	0m 8s	master passed
-0 ⚠️	patch	0m 18s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 12s	the patch passed
+1 💚	compile	0m 11s	the patch passed
+1 💚	javac	0m 11s	the patch passed
+1 💚	checkstyle	0m 5s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 6s	the patch passed
		_ Other Tests _
+1 💚	unit	3m 30s	hbase-hbck2 in the patch passed.
+1 💚	asflicense	0m 7s	The patch does not generate ASF License warnings.
		6m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/5/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#81
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux 272a241a52a7 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `90e0f45`
Default Java	Oracle Corporation-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/5/testReport/
Max. process+thread count	1200 (vs. ulimit of 5000)
modules	C: hbase-hbck2 U: hbase-hbck2
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/5/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-08-20T02:29:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+0 🆗	spotbugs	0m 1s	spotbugs executables are not available.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1 💚	mvninstall	1m 1s	master passed
+1 💚	compile	0m 10s	master passed
+1 💚	checkstyle	0m 12s	master passed
+1 💚	javadoc	0m 7s	master passed
-0 ⚠️	patch	0m 23s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 11s	the patch passed
+1 💚	compile	0m 9s	the patch passed
+1 💚	javac	0m 9s	the patch passed
+1 💚	checkstyle	0m 4s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 6s	the patch passed
		_ Other Tests _
+1 💚	unit	3m 15s	hbase-hbck2 in the patch passed.
+1 💚	asflicense	0m 6s	The patch does not generate ASF License warnings.
		6m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/6/artifact/yetus-precommit-check/output/Dockerfile
GITHUB PR	#81
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux c19d22702a6a 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 GNU/Linux
Build tool	maven
git revision	master / `90e0f45`
Default Java	Oracle Corporation-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/6/testReport/
Max. process+thread count	1185 (vs. ulimit of 5000)
modules	C: hbase-hbck2 U: hbase-hbck2
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-Operator-Tools-PreCommit/job/PR-81/6/console
versions	git=2.20.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Mengqi777 · 2021-08-20T02:29:40Z

@wchevreuil hi bro, thanks for your suggestion, I have change the name to reportundeletedregionsinmeta and add the UTs, PTAL.

Apache-HBase · 2022-02-07T04:55:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	patch	0m 2s	#81 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/in-progress/precommit-patchnames for help.

Subsystem	Report/Notes
GITHUB PR	#81
Console output	https://ci-hbase.apache.org/job/HBase-Operator-Tools-PreCommit/job/PR-81/1/console
versions	git=2.25.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

mengqi added 6 commits December 1, 2020 14:07

remove version check

61a9cd6

Merge branch 'master' of https://github.com/apache/hbase-operator-tools

6aa33a0

report dirty metadata in meta table

d0c9b6f

recovery version check

60adca4

recovery checkFunctionSupported

9e05d90

format code

8b26d09

wchevreuil requested changes Aug 16, 2021

View reviewed changes

change dirty region to undeleted region, add undelete region report UTs

6a974ae

mengqi and others added 4 commits August 20, 2021 08:44

Merge remote-tracking branch 'upstream/master'

0813367

Merge branch 'apache:master' into master

e6caab1

get row bytes from result

c1a7b39

Merge branch 'master' of https://github.com/Mengqi777/hbase-operator-…

3ed3973

…tools

fix check style and fix TestHBCK2#testReportUndeletedRegionsInMeta

d4ea548

mengqi added 2 commits August 20, 2021 10:12

remove err describe

121908c

update usageReportUndeletedRegionsInMeta

a7482dc

Mengqi777 requested a review from wchevreuil August 23, 2021 08:51

Conversation

Mengqi777 commented Feb 9, 2021

Uh oh!

Apache-HBase commented Feb 9, 2021

Uh oh!

wchevreuil left a comment

Choose a reason for hiding this comment

Uh oh!

wchevreuil Aug 16, 2021

Choose a reason for hiding this comment

Uh oh!

wchevreuil Aug 16, 2021

Choose a reason for hiding this comment

Uh oh!

wchevreuil Aug 16, 2021

Choose a reason for hiding this comment

Uh oh!

wchevreuil Aug 16, 2021

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Aug 19, 2021

Uh oh!

Apache-HBase commented Aug 20, 2021

Uh oh!

Apache-HBase commented Aug 20, 2021

Uh oh!

Apache-HBase commented Aug 20, 2021

Uh oh!

Apache-HBase commented Aug 20, 2021

Uh oh!

Mengqi777 commented Aug 20, 2021

Uh oh!

Apache-HBase commented Feb 7, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants