Mountain landscape during sunset

Filling in the Details

Since our study began in 2018, we’ve been thinking big: beginning with a blank slate and asking broad questions about what the bar exam can and should be. Now, it’s time to start filling in the details. The Task Force has reached a pivotal moment in its three-year study: the Blueprint Development Committee (BDC) and the Test Design Committee (TDC) have completed their work for Phase 3.

The Task Force’s instructions to both committees were: (1) Base your work on the data collected during the Phase 1 listening sessions, where over 400 stakeholders provided input, and the Phase 2 practice analysis, where nearly 15,000 lawyers provided ratings of the frequency and importance of job-related tasks and knowledge, skills, and abilities; and (2) Focus on the purpose of the bar exam, which is to protect the public by helping to ensure that those who are newly licensed possess the minimum knowledge and skills to perform activities typically required of an entry-level lawyer. The Task Force’s independent research consultant, ACS Ventures LLC, guided the committees through their work.

The BDC met virtually over three days at the end of June to review the results of the practice analysis and consider the  job-related tasks, knowledge areas, and skills that were rated as being performed Frequently or Moderately by 50% or more of the survey respondents, as detailed in the Testing Task Force Phase 2 Report: 2019 Practice Analysis. The 17 lawyers who served on the BDC were drawn from 13 jurisdictions, a range of practice settings (private law firms, government, in-house, nonprofit organizations, legal services/public interest, and judicial law clerk), various types of location (rural/small town, urban, and multi-jurisdictional), and 20 different practice areas. Additionally, the BDC was diverse in composition, with 14 women and 11 people of color.

The BDC thoughtfully considered each of the tasks and knowledge areas and discussed what should and should not be tested on the next generation of the bar exam based on their collective knowledge, experience, and understanding of the purpose of the bar exam. The BDC also considered how much emphasis should be given to assessing application of skills in a content-dependent context (knowledge of law is necessary) versus a content-independent context (legal resources are provided in the test materials). They recommended that approximately 60% of the exam be devoted to content-dependent skills (knowledge of law) and 40% devoted to content-independent (lawyering skills). The BDC endorsed testing the following knowledge areas, with some areas receiving more emphasis or weight than others, and skill domains:

Knowledge Areas Skill Domains
Civil Procedure Legal Research
Contracts Legal Writing
Criminal Law & Procedure Issue Spotting and Evaluation
Evidence Investigation and Analysis
Torts Client Counseling and Advising
Real Property Client Relationship and Management
Constitutional Law Negotiation and Dispute Resolution
Business Organizations

 

Once the BDC had completed its work, the TDC met virtually to consider how to best measure the knowledge and skills identified by the BDC. The TDC was composed of 28 bar administrators, bar examiners, justices, and legal educators from 20 jurisdictions and 11 law schools, including 10 women and seven people of color. The TDC discussed the pros and cons of various test design considerations such as structure, assessment method, score interpretation and use, and accessibility and fairness; the various perspectives on the issues were influenced by the unique role each member has in the education and licensing of newly licensed lawyers. ACS used the discussion points from the first two days of the TDC meeting to develop a few design models for the TDC members to review independently and provide feedback about via an online survey before reconvening for a third day to discuss the models and achieve consensus on various design issues.

While we’re still finalizing the report of the TDC’s work, which we hope to share within the next couple of weeks, one design element that we can share at this point is that the next generation of the bar exam will be delivered on computer—either at testing centers or on candidates’ own computers in an in-person setting or remotely.

Early this fall, the TTF will be seeking reactions from all stakeholders to the blueprint and design recommendations proposed by the BDC and TDC. Although a lot of work remains to be accomplished before the Task Force makes its final recommendations to NCBE’s Board of Trustees at the end of this year, the finish line is in sight. We extend our deep appreciation and thanks to all the members of the BDC and the TDC for their time, hard work, dedication to the legal profession, and invaluable contribution to building the next generation of the bar exam!