About the Testing Task Force
NCBE studies many aspects of the bar examination on an ongoing basis. Periodic confirmation that the content being tested on a licensing exam aligns with competencies expected of a newly licensed professional is common and prudent in the field of licensing exams. The last large-scale survey-based study of this nature undertaken by NCBE was conducted via a job analysis in 2012. The Testing Task Force reflects NCBE’s ongoing efforts to support jurisdictions and to sustain confidence by all stakeholders in the testing process.
The Task Force study will be more comprehensive than the 2012 job analysis in several ways. The 2012 study identified the job activities of newly licensed lawyers in sufficient detail to provide a job-related and valid basis for the development of licensing examinations offered by NCBE. The Task Force study will identify competencies required of newly licensed lawyers in the 21st century; it will also research whether, how, and when the identified competencies could be assessed and in the process will study test formats, test delivery methods, and the timing of test administration.
The legal profession, like other professions, is undergoing a period of accelerated change. Some of the changes are brought on by rapid developments in technology that impact both the delivery of legal services and the types of legal problems for which clients seek representation. Technology is also contributing to advances in the science of assessment and testing. In addition, competency modeling in the legal profession has become increasingly important in a variety of contexts, including assessment of competencies expected of newly licensed lawyers to protect the public. It is prudent to undertake a comprehensive and systematic study at this time to ensure that the bar exam keeps pace with developments and changes in the practice of law, legal education, technology, and testing.
The Task Force’s independent research consultants—ACS Ventures, LLC, and American Institutes for Research (AIR)—will engage with stakeholders throughout the study. Our consultants will use focus groups, forums, and surveys to gather input about the bar examination and about changes and trends taking place in the profession that impact the work of newly licensed lawyers.
The Task Force has selected two independent research consulting firms—ACS Ventures, LLC, and American Institutes for Research (AIR)—to support its study. In addition, the Task Force’s work is supported by NCBE’s Technical Advisory Panel of measurement experts.
Members of the Task Force bring many years of combined experience from their roles serving on the bench, in private practice, on state boards of bar examiners, as bar administrators, as legal employers, and as legal educators. They are familiar with NCBE’s examinations and knowledgeable about its test development processes. The Task Force is purposefully small in number to permit deliberate execution of its work, but it is committed to a comprehensive and collaborative study that draws heavily on independent expert assistance and on outreach to and input from stakeholders.
The Task Force will oversee the study, which will include gathering input from stakeholders. The Task Force will periodically report on its progress to the NCBE Board of Trustees and through updates on its website. Ultimately, the Task Force will prepare a final report setting out its recommendations. The report will provide a documented record of the Task Force’s work and will include qualitative and quantitative data that support its findings and recommendations.
The Task Force plans to complete its work and submit its report to the NCBE Board of Trustees by the end of 2020.
The Task Force is approaching this study with no preconceived conclusions about whether the bar exam will or should change, whether in terms of content, format, timing, or method of delivery. Recommendations made by the Task Force will be driven by empirical research and stakeholder input. If any changes to the bar examination are recommended by the Task Force, all stakeholders, particularly state bar admission authorities and legal educators, will be informed about the nature of and the rationale for the proposed changes and will be given a meaningful opportunity to provide input on those changes. As has always been the case, no changes will be made to the bar exam without affording ample notice.
The Uniform Bar Exam (UBE) is not a test product. Rather, it is a compact among jurisdictions that agree to use the same exam components, to adopt a set of policies to produce portable bar exam scores, and to accept the transfer of those scores from other UBE jurisdictions. The Task Force’s study will not change the premise on which the UBE is based—that use of the same exam components, administered and graded uniformly, produces bar exam scores that can be relied upon by other jurisdictions, sparing applicants from sitting for the bar exam in each jurisdiction in which they seek admission. UBE jurisdictions are stakeholders whose input will be sought and encouraged as the Task Force undertakes its study.
Setting the passing score for licensure is the prerogative of each jurisdiction. As such, the Task Force does not plan to include a standard-setting study within the scope of its work. However, research on score conversion or similar topics may be conducted if the Task Force’s study warrants it. NCBE will continue to provide jurisdictions with guidance regarding the appropriate interpretation and use of test scores.
The Task Force looks forward to receiving feedback and input from other stakeholder groups that are studying the bar exam. Although the Task Force study will be undertaken independently, the Task Force plans to share the results of its research with other interested stakeholders.