Problems and countermeasures in natural language processing evaluation

Because of the guiding significance of NLP evaluation, it will seriously affect the fairness and objectivity of work evaluation in related fields if the evaluation itself has deviations.New evaluation data sets and tasks have been continuously proposed in recent years.

However, the existing evaluation also exposed a series of problems in terms of scientificity and objectivity. Inappropriate evaluation will limit the advancement of natural language processing technology. This report will analyze the current situation and existing problems of NLP evaluation, as well as propose assumptions and prospects for future NLP evaluation.