Tongue Image Database Construction Based on the Expert Opinions: Assessment for Individual Agreement and Methods for Expert Selection

Table 3

Improving process of the interrater agreement for “the moderate tongue texture.”

Recognition times	Identified discordant experts	Rest experts	Inter-rater agreement for the rest of the experts
1	rater 45, rater 49, rater 32, rater 42, rater 34, rater 40	44	0.1674
2	rater 20, rater 16, rater 13, rater 3	40	0.2220
3	rater 5, rater 43	38	0.2470
4	rater 15, rater 25	36	0.2707
5	rater 29, rater 39, rater 33, rater 47	32	0.3247
6	rater 31, rater 35, rater 50, rater 26, rater 38	27	0.4098
7	rater 41, rater 17, rater 1, rater 10	23	0.4890
8	rater 46, rater 19, rater 44, rater 37, rater 30, rater 18, rater 11	16	0.6333