Law School Exam/Grading System

Post Reply
Twombly
Posts: 13
Joined: Fri Mar 23, 2018 11:06 am

Law School Exam/Grading System

Post by Twombly » Wed Apr 25, 2018 9:49 am

For the vast majority of students and especially those attending law schools just outside the T14, grades matter a lot. They will have a significant impact on your career. Most can agree that to some extent having poor/median (wherever you want to draw the line) grades at a non-T13 school (and even some T13 schools) will almost certainly close off many opportunities for a lawyer, especially immediately post-grad. Sure, grades have a diminishing importance over the span of your career but inevitably they can close off or open up certain opportunities in the first few years after you graduate. Arguably, what you do in the first few years after you graduate will in turn close off or open up other opportunities, etc. So bad grades can really start you off on the wrong or less-than-ideal track.

Because grades have such a fundamental importance and are the main (sometimes only) tool for sorting applicants, we would assume that the way in which grades are ultimately assigned would reflect the level of importance that grades have. Furthermore, it seems well accepted that your 1L grades matter more than any other year.

My main question is then, considering all this, is there even a shred of evidence that the traditional system of exams and grading works to distinguish between and adequately sort out the "good", "great" and "bad" students/soon-to-be-lawyers. Whatever qualities we want in a "great" lawyer (hardworking, intelligent, writing abilities, whatever) - does the current system of giving one lengthy exam at the end of the semester truly accomplish this sorting? That grades really do distinguish the good, great, and bad?

If not, then why do law schools still use it? I understand tradition and that this system was in place when the professors were in law school - but if there is in fact a more superior method, what are the arguments for following tradition?

OR, perhaps the argument is that the reason this method of grading is still around is because it really does accomplish this sorting between great, good and bad. But where is the evidence for that?

User avatar
Nony
Posts: 5557
Joined: Thu Jan 25, 2018 2:34 pm

Re: Law School Exam/Grading System

Post by Nony » Wed Apr 25, 2018 10:02 am

Schools use the kind of grading system they do because it facilitates ranking students, which lets employers (the ones who care about grades) decide who to hire. The relationship between ability to succeed at taking law school exams and being a good lawyer is pretty tenuous and not really connected to the material (i.e. reading/understanding quickly, thinking quickly, being able to write quickly in a way that comprehensively addresses what’s in front of you are generally proxies for intelligence and are relatively useful skills for succeeding in a lot of jobs, but not really specific to law or the law school grading system).

But there isn’t much point in asking why law schools use it. They do, and that is exceedingly unlikely to change, except perhaps incrementally over a long period.

User avatar
UVA2B
Moderator
Posts: 3239
Joined: Thu Jan 25, 2018 3:26 pm

Re: Law School Exam/Grading System

Post by UVA2B » Wed Apr 25, 2018 10:11 am

Nony pretty much covers my thoughts on these questions, but in order to slightly answer the question, the system does a pretty solid job of grouping the great law students (note I’m not saying great future attorneys, just people who are good at taking law school exams), the bad law students, and putting the rest in a hulking middle of The class. The truth is, at least in my opinion, that most students are not demonstrably better at taking exams/issue spotting and analysis, and few are notably great or bad, so the forced curve serves its purpose: differentiate the adept law school exam takers and the worse law school exam takers, while leaving the rest to be compared on the rest of their merits by potential employers.

That said, while some opportunities are closed off if you don’t have a high GPA, that doesn’t necessarily mean you can’t or won’t find the fulfilling career you hopefully want.

lolwat
Posts: 675
Joined: Mon Jan 29, 2018 2:06 am

Re: Law School Exam/Grading System

Post by lolwat » Wed Apr 25, 2018 8:57 pm

Overall, I'd agree with Nony and UVA. I didn't like the disparity with grading among different sections at my school, though. It's all on the same forced curve, but some professors compressed the curve so the "top" grade in that section might be significantly lower than the top grade in another section of the exact same class. So you could get the highest grade in your Torts class and still have a grade below like 10 other people from other sections.
That said, while some opportunities are closed off if you don’t have a high GPA, that doesn’t necessarily mean you can’t or won’t find the fulfilling career you hopefully want.
The way this is phrased is certainly true. I do think many people probably never reach whatever goal they shot for, but rather become "fulfilled" more by learning to accept and be happy with whatever they get/end up in. Or they lateral until they're somewhere they can be happy. Or they leave the law. :)

User avatar
Nony
Posts: 5557
Joined: Thu Jan 25, 2018 2:34 pm

Re: Law School Exam/Grading System

Post by Nony » Wed Apr 25, 2018 9:37 pm

Eh, a lot of people go to law school to be PDs/ADAs and get hired by places that don't even ask for grades.

Twombly
Posts: 13
Joined: Fri Mar 23, 2018 11:06 am

Re: Law School Exam/Grading System

Post by Twombly » Thu Apr 26, 2018 3:37 pm

These responses to my question are not quite directly responding to what I am trying to ask. I understand that grades are for employers to sort through applicants, that poor grades don't mean you won't find a fulfilling career or that the chances that a law school will change their approach is slim. I'm not getting at any of that.

I'm more so attempting to ask - make yourself the Dean of a good law school. A school that wants to move up, be competitive, and is already well respected. (This eliminates lower ranked schools that will try gimmicky stunts solely to stand out). Now imagine you have the power to restructure how the law school exam and grading system works. You think, what is the ultimate purpose of our grading system? Well, for us to rank our students in a way that employers are able to efficiently sort through students and determine who they want to hire. Presumably, the employers will want at a minimum students who will be good/great lawyers and employees. We should make sure our grading system is doing a good job at providing that assurance to an employer.

Now the next question is, does our current system actually accomplish that? Not "does our current system distinguish good-exam-takers from bad-exam-takers" but does it actually speak to their potential as a lawyer in a reliable way? Maybe the answer is yes, it does a decent job. However, why would we not try to develop an even better system? What are the reasons why we keep this one?

Is there no better system and we are working with the best idea out there? (I find this doubtful). This all seems very logical to me. Why wouldn't an institution, that essentially exists to train people to do a specific job and then send them out to do it, not want to develop the best methods possible?

I imagine in some academic fields, if the only argument you have is "this is how we have done it for X number of years and it's just the way it is" you would not be taken seriously.

Again, these questions might not have answers. And that might be my point. Have there been any studies or any research to answer the basic question - does the traditional law school exam system actually work well enough to accomplish its purported goals to justify not changing it? And if not, what exactly is keeping a law school from changing beyond commitment to tradition? It seems to me that it would benefit a law school to be able to say, we have concrete evidence that our system of grading applicants is far superior at distinguishing between our students, therefore employers can have greater confidence that when they hire a student from our school with good/great grades, they are getting a good/great attorney. Obviously no system will be flawless, but one system can certainly be more reliable than another.

User avatar
Nony
Posts: 5557
Joined: Thu Jan 25, 2018 2:34 pm

Re: Law School Exam/Grading System

Post by Nony » Thu Apr 26, 2018 4:21 pm

When hiring by employers who care most about grades (i.e. biglaw and government) is premised on the current grading system, inertia is a huge thing, and being the school that sticks its neck out to change can hurt a school with employers. Also alternative grading systems are frequently more labor intensive for profs, who succeed based on research, not teaching, so (collectively) have no incentive to change (although there are many individual conscientious profs dedicated to teaching).

There are probably lots of people involved with law schools who think it’s a perfectly fine system, too. A lot of STEM fields grade on strict curves. As I suggested earlier, law school exams tend to measure ability to read quickly, analyze quickly, and formulate a clear response quickly. Those are usually fairly useful skills. Students who “get” how to take an exam quicker than their counterparts could be law school exam savants, or they could be more adaptive/learn quicker, which would also be attractive to employers.

What kind of alternate grading system are you thinking about? I don’t think the one-exam-and-done is an effective way for students to learn, but I think it’s an excellent way to rank who will be able to grind it out in biglaw for a few years. I think it’s probably not very good at measuring who can walk into a courtroom and defend a client at a criminal trial, or who can develop a book of business for a firm. But I don’t know of a grading system (in a doctrinal course) that would measure that. Do you have an alternative system in mind?

(Keep in mind, too, that schools are pretty dedicated to the idea that they teach you to think like a lawyer and that there is something distinctive about this. Culturally there’s a strong belief in the connection between thinking like a lawyer and the style of teaching used in law schools.)

lolwat
Posts: 675
Joined: Mon Jan 29, 2018 2:06 am

Re: Law School Exam/Grading System

Post by lolwat » Thu Apr 26, 2018 10:46 pm

I’m on my phone and can’t post a lengthy response but I’d ask what’s your definition of a good/great lawyer or employee? There are tons of skills involved and I believe the current grading system is fine overall for the same reasons Nony noted above. A lot of situations involve being to analyze the situation and present the analysis in a coherent manner... and to do so quickly. You may have to do that with facts rather than the law, you may have to do that in oral argument rather than written work, you may have to do that in a courtroom rather than at your office, and you may have to do that in front of a jury rather than a judge—but many lawyering skills still come down to a core of being able to analyze, think, and respond quickly to something.

I also think the fact that many schools offer practical skills courses which grade students differently than the typical law school exam helps give a more well-rounded picture too if employers are interested in that.

bruh
Posts: 600
Joined: Fri Jan 26, 2018 9:04 pm

Re: Law School Exam/Grading System

Post by bruh » Thu Apr 26, 2018 11:00 pm

Grades are still the best proxy for being a good litigator in complex litigation, even if it's a really, really, really, really, really, really rough one. Whether that's true for transactional associates is beyond me.

User avatar
Finn
Posts: 375
Joined: Fri Jan 26, 2018 2:59 am

Re: Law School Exam/Grading System

Post by Finn » Thu Apr 26, 2018 11:10 pm

I think a lot of it has to do with the (in)ability to define a good/great attorney in a qualitative or quantitative way for possible research that would try and answer your question(s). Let alone the politics of whether a school's independent research will be perceived as objective or significant by employers or peers. A school may not view the ability to say "we have concrete evidence that our system of grading applicants is far superior at distinguishing between our students" as worth the risk. For a school to offer "concrete" evidence and prove "superior" results, that school would have to take years and resources that may ultimately not be worth the fuss if employers/peers challenge or reject the research.

Twombly
Posts: 13
Joined: Fri Mar 23, 2018 11:06 am

Re: Law School Exam/Grading System

Post by Twombly » Fri Apr 27, 2018 11:56 am

I'm not sure how changing the way grades are assigned/determined would really affect employers? It seems to me as long as an applicant shows up with 3.XX GPA on their resume, the employer isn't going to care one wit whether you got that GPA by only taking 48-hour take home open book exams, 3 hour closed book exams, wrote a bunch of massive papers, or whatever other method of assessment was used. (I use these just as examples, not suggestions.)

A small example is if a school required all 1L classes to take 3 hour closed book exams and then decided that in fact, 48-hour take home exams better assess the student's skills and so they shift to require all 1L classes to now use the 48-hour model, employer's won't bat an eye or even notice. Perhaps I'm wrong, I am not sure, but I can't imagine employers caring that much beyond "what's this applicant's GPA?"

I'm not suggesting we get rid of grades. I'm asking why the method of assessment used to ultimately assign that grade hasn't really changed beyond the traditional model of one giant exam at the end of a semester (at least for doctrinal and thus most 1L classes which is arguably your most important grades)

Yes, I see everyone reiterating that the current exam system assesses your ability to quickly analyze facts, etc. etc. But has there been any research to actually prove this? I can't find any. I know it sounds very logical and intuitive, but lots of things that seem intuitive can turn out to be wrong after someone does some experiments, research (or whatever it takes - you get what I mean).]

For example, as everyone knows, Harvard is now accepting the GRE alongside the LSAT. I have no doubt there are lots of reasons motivating this change, but also this:

"The change is supported by an HLS study, designed in 2016 and completed earlier this year, examining, on an anonymized basis, the GRE scores of current and former HLS students who took both the GRE and the LSAT. In accordance with American Bar Association (ABA) Standards for Legal Education, the aim of the study was to determine whether the GRE is a valid predictor of first-year academic performance in law school. The statistical study showed that the GRE is an equally valid predictor of first-year grades."

Has any school said, hey let's see if our current examination system is a valid predictor of being a good attorney? Or do they just assume that it is? Am I the only one that finds it kind of crazy that the answer seems to be no, no school has done that?

I don't think that the criteria that make a good attorney is so obscure that it would be impossible to design a decent study. Furthermore, if not an independent school then why not the ABA or a group of schools getting together in a joint effort create "best practices" backed by evidence for their policies and methods used?

Generally overall I just suspect that a lot of things done in law schools aren't done because there is good evidence to back them up, and that seems crazy to me in the 21st Century that there isn't any schools pioneering new ways of doing things.

User avatar
Nony
Posts: 5557
Joined: Thu Jan 25, 2018 2:34 pm

Re: Law School Exam/Grading System

Post by Nony » Fri Apr 27, 2018 12:14 pm

Okay, so the method of assessment isn’t the same as the grading system (for instance, grading on a curve), which I think is where people were getting confused.

Also, there is a whole industry of education profs who study exactly this kind of thing. I haven’t researched it myself, but I promise you that there are people out there who have. Schools debate this kind of stuff a ton. I don’t think there’s no research out there, it’s a question of where to look. (It may not be described as specific to law school, but there’s TONS of discussion of exams v. papers v. presentations etc.)

The difficulty is 1) defining what makes a good attorney. I would say that the one-exam-at-the-end doesn’t actually test a ton of stuff that’s really important for success as an attorney in many fields, primarily things like organizational ability and people skills. But I also don’t know how a law school would test for this, and not all jobs require these things in the same degree. There are lots of ways to be a great attorney. So it’s hard to agree on what you should measure 2) agreeing on how to measure that. Even in higher ed generally there are tons of different approaches. Some profs lecture and give traditional exams. Some profs have people write and talk. Some profs arrange a course around hands on projects. Some profs use labs. Etc. Each method has its passionate supporters and equally passionate detractors. 3) overcoming the generations of culture and history behind the unique law school approach. The Socratic method and the one exam thing with outlining is a huge part of how legal education defines itself as unique. 4) as noted, people working in legal academia have lots of other interests competing with student learning. 5) because law is a professional program and preparing people for licensure, there is a greater expectation that curricula be relatively uniform across schools. When everyone has to teach torts, contracts, etc to 1Ls, and meet ABA accreditation requirements, there’s a lot of incentive to do what everyone else does. This is why you’ll see a lot more variation after 1L (especially with clinics and other practice-based courses).

So I promise you there are people talking about and studying this. It’s just that studying it and implementing changes are two different paths.

User avatar
Finn
Posts: 375
Joined: Fri Jan 26, 2018 2:59 am

Re: Law School Exam/Grading System

Post by Finn » Fri Apr 27, 2018 12:53 pm

Personally, I don't think there is an empirical way to "prove" that one method of assessment is better than another in the context of law school. For example, some contract exams are MC, some are not. There's no reason to doubt that some students would prefer one exam over the other or that certain students would excel in one kind of exam over the other. However, to claim one approach is objetively superior would be hard to accept let alone when it challenges the various pedagogies and philosophies professors have on administering these exams.

Even in the example of the take-home exam. In parallel universes, where the exact same class takes a 3 hour closed book exam in one reality and a 48-hour take-home exam in another, a statistically significant amount of reiterations doesn't really provide us with results that can be easily implemented in law schools. Either the same students get the exact same grades in both, students fluctuate slightly within their As and Bs, or the folks at the bottom in one exam get the top grades in the other exam and vice-versa. Nonetheless, whatever this thought experiment offers, what it does not help is getting to (I think) your point that folks who know the material and have the potential to be fantastic lawyers will not necessarily be at the top of the class every time. Therefore, returning from this parallel universe, to objectively say one method is more effective at assessing students would be extremely difficult to prove and yet still somewhat irrelevant when folks are still on a forced curve that doesn't really take into consideration a student's skills but rather a student's skills during that particular exam in comparison to the student's peers in that particular exam.

At the end of the day, you're not alone in asking these questions but (and Nony really nailed it here and I'm only reiterating this because I think she's getting to the crux of your questions) studying it and implementing changes are two different paths.

User avatar
pancakes3
Posts: 3361
Joined: Thu Jan 25, 2018 3:00 pm

Re: Law School Exam/Grading System

Post by pancakes3 » Mon Apr 30, 2018 1:11 am

the question re: grading via testing is not unique to law school, so that's pretty much unanswerable.

the question re: classes in general may be more answerable (shifting it to focus more on legal methods than legal doctrine; clinics over lectures, etc) is more unique to law school, but still one that plagues many professional degrees (medicine probably has a bigger gripe than law) and similarly unanswerable.

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests