This year we have seen some ups and downs with scores which have reflected a greater number of opinions on the site and a much greater geographical distribution of reviewers. On top of that we have a more wide-ranging coverage of releases.
Individual writers have debated the merit of scoring, commenters have challenges scores based on their own considered quality, influencers have written about dropping scores due to misinterpretation of these scores. I personally reflect hard on the scores I give before each published review and continue to go back to thinking about my benchmark dram which scored a 6/10. When I score anything else six out of ten I reflect on whether it deserves to be aligned with such a fine dram; when I score above I think about what qualities elevate the dram above that dram. Once you get into 9s or 10s, the whisky really must stop you in your tracks and demand your attention.
It has been established that presentation, environment, prior expectations, experience, mood, season, time of day, misconceptions and diet can all affect our experience of a whisky and therefore the ratings we provide. Some of the routines and rituals that tasters and reviewers go through prior to and during a whisky tasting can remove some of these factors, others remain more intangible.
Photo courtesy of Master of Malt.
Then we have to consider price, which may or may not result in the deduction of points, along with other, less-often-used factors that may result in points being added or subtracted.
Nevertheless, I propose for discussion a scoring grid that goes beyond the “we use the full 10 points of the scale” statement and help people understand and comprehend why 6 means a great dram and does not translate to 60/100 on a 100 point scale, which only really uses 20 points between 75 and 95 in practice. The descriptors should assist and support the scorer and give clarity to those who are reading the review.
I have produced it below and followed with some indicative tasting notes from previous reviews, and a few additions to illustrate the point. I’d love to hear from readers and Malt writers about the usefulness of the descriptors and any suggestions. Perhaps we could reach a broad consensus? We’ll never agree on scores, but at least we can understand what each score really means to the reviewer.
Click here to see a high resolution version of this chart.
Fujikai Japanese Single Malt Whisky
43% ABV; about £50 in 2015. Photo Courtesy of Master of Malt.
On the nose: Window putty, rubber squash balls, chemical cupboards, wood shavings, ozone, permanent marker, hot lamination machine, Crayola crayons.
In the mouth: Sharp, jarringly unnatural, hairspray, contact adhesive, paper ash, damp newspapers, new car tyres. Finish is slightly solvent driven and vegetal.
Conclusions:
Certainly deserving of 1. It’s probably not whisky at all, as it is rumoured to be aged rice spirit rather than malted barley. It is barely palatable and tastes rather like some cleaning chemical was accidently sprayed into your mouth. Interestingly, the week I complied these notes I sent out 2 5cl samples of this whisky to people who had heard about this legendary fail.
Score: 1/10
Kingsbarns Cask 1610869 – Review
Bourbon barrel. 8/9/16 to 2019. 61.7% ABV. £75.
Colour: Pale straw
On the nose: Strong bourbon influence of vanilla and light toffee, crisp green apple, lemon peel, a richer weightier note developing too, some riper orchard fruits, soft brown sugar, and apple blossom. With water the fruit is elevated as are the more floral blossom notes.
In the mouth: A little tight at full strength, but again some weightier flavours here. Honeycomb, vanilla fudge, some oak notes giving quite a bitter finish. Still quite tight with water, coming across as quite young; unwaxed lemons, peppery heat, foam shrimps, still bitter.
Conclusions:
The nose hints at the development of flavours but this has been bottled too young. I’m not exactly sure why this barrel was chosen as a showcase single cask in 2019. Not really any enjoyment in this for me.
Score: 3/10
Cadenhead’s BenRiach 2010 – Review
10 years old. Cognac Butt Finish for one year. 57.1% ABV. £45. Released as part of the shop collection (Edinburgh) in 2010; sample provided by the Whisky Sleuth. Image courtesy of The Whisky Shop
Colour: Pale gold.
On the nose: Glazed danish pastry, madeleines, bread and butter pudding, really rich, almost sherry-like, toasted sugar almonds, and a slight funk of bruised orchard fruit.
In the mouth: Good body, sweet and spicy from both wood and spirit, more sugar notes, golden syrup, very sweet, more chilli heat with youthful spirit, oaky, drying finish with some soft baking spices.
Conclusions:
Just a bit too sweet for me. I can see that there is some influence from the cognac cask but not enough to save this dram; maybe one of these less active casks that has been well used before having whisky in or maybe the spirit needed more time. The price should be acknowledged as very fair and therefore…
Score: 5/10
Oddbins Tomatin Whisky Meets Sherry – Review
53.4% ABV. £25 for 350 ml. Photo courtesy of Oddbins.
Colour: Light Oloroso sherry.
On the nose: An initial puff of spirit quickly followed by deep rich sweetness; warm dates; ginger loaf, and then light citrus and distant peach.
In the mouth: Sherry — distinctly, good sherry — and an alcohol burn; some maltiness, and a nutty finish. The full-strength spirit cuts through the richness of the sherry nicely. There is a gentle bitterness on the finish. A decent slug of water greatly improves both the nose, mouthfeel, and finish. Tongue-suckingly delicious.
Conclusions:
This dram is a lesson in good sherry and decent whisky being brought together for the right amount of time. It’s not multi-dimensional; the spirit does not sing through the sherry and elevate it to another level. This is a respectful 50-50 marriage of flavours. It’s keenly priced, tasty stuff, and a well sized bottle for an experiment too. I understand the last few bottles are still available.
Score: 6/10
Loch Lomond 12 years old – Review
46% ABV. £35. Photo courtesy of Loch Lomond.
On the nose: Smooth sherry and gentle peat, rich dessert syrup, raspberries, gummy strawberries, dried ginger, nutmeg, cinnamon. Slightly ashy with a hint of leather.
In the mouth: Smooth and juicy, gentle wood spices, mesquite chilli, smooth and fruity again on the finish with some peat spice and cask giving a remarkably length of spirit for a whisky of this price.
Conclusions:
One of the earlier drams I tried on the evening, and it certainly took me by surprise. There is a lot going on here. I am pleased that, at the time of writing these notes, I am still really salivating over this. I could compare this to McNair’s Lum Reek 12 year old blend at over £50, perhaps to help demonstrate the value for money here. It says “Perfectly Balanced” on the bottle and it’s certainly a session whisky.
Score: 7/10
Oban 12 Years Old 2021 Special Release – Review
From freshly charred American oak casks. 56.2% ABV. £105.
Colour: Pale gold.
On the nose: Grassy and herbal, light butter toffee, vanilla, meringue, sugary and sweet, no char detectable on the nose.
In the mouth: Bang, super fruity and rich, pick’n’mix jellies, fizzy sour sweets, M&S Colin Caterpillars, getting more complex on the finish with oak spices, a hint of charr, blood orange and a pinch of salt on the extra-long finish.
Conclusions:
I’d say this has a lot going on; best taste profile too. Plus, it’s pretty hard to get Oban at cask strength sadly. At the distillery the bottle-your-own is only 20cl, and many whisky enthusiasts have gone right back for a second bottle of that having tasted it. This will be very popular.
Score: 8/10
Gordon & MacPhail Port Ellen 40 Years Old (1979-2019) – Review
54.7% ABV. SRP of £2,495. Photo courtesy of The Whisky Exchange.
Colour: Pale gold.
On the nose: Gentle peat intermingled with delicate fruits; actually the peat is more prominent that I expected at this age. Very old varnish, elastic bandage, BBQ charred beef fat, a light fruitiness bringing it all together.
In the mouth: Delightfully fruity at first, soft ashy peat and wood spice, nutmeg and allspice, then mineral notes, a slight waft of earthy autumnal bracken, more sweet wood spices, peaty chimney smoke on the breeze, oak influence growing alongside the peat which dances across the middle of the tongue on the finish.
Conclusions:
What an incredible whisky, which has been opened and shared. Kudos to CJ at Single Malt Vault for that from the outset! This is a whisky from a time before the internet, when Islay was a remote and unknown place, before the influencers arrived. Liquid history that cannot be googled or found on Wikipedia.
I was recently asked the criteria for giving a high score and I said it would have to be whisky I could “get lost in.” This is certainly one of those, a beguiling nose rewards patience and time. The palate surpasses the nose, which is often not the case with modern whisky. It is surprisingly well balanced and not too woody despite 40 years in a cask. It is a testament to Gordon & McPhail’s cask management for eking out these barrels to such a ripe old age.
Score: 9/10
Lead image courtesy of BBC.
Ahh a subject that has some similarities to my work as a commodity pricing editor.
As long as you guys have a clear scoring methodology (which you do, but maybe don’t post it as a screenshot because the words become blurred when I zoom in), and all reviewers follow them, us readers shouldn’t have any complaint (or at else I won’t).
For eg, my scoring methodology is slightly different from yours, but since I’m aware of the differences, I can make an educated guess on where you will stand on a particular bottle you’re scoring.
I agree with you on the point revision with regards to price, my only suggestion is to give a few additional scoring for bottles that are very difficult to find at retail pricing. This is helpful to me when deciding whether it’s worth it to buy a bottle above the MSRP.
A good example will be Taylor’s review of the George T Stagg 2017. He gave a score of 10/10 based on the $300 that he paid for it. But the average reader will probably be able to find it at only the secondary value. So it will be helpful to me if Taylor estimate the score he might give if he bought it was $1000 (or whatever the street price is).
Thanks for your comments. I certainly think we need to engage with the realities of the secondary market.
We’ll see what we can do regarding resolution of the scoring guide too.
I agree with ALMOST all of your scoring criteria but do not agree with including cost in the scoring. Once the whisky is bottled and released, the flavor profile is set and will remain consistent for its life in the bottle unless it spoils from poor handling/storage.
Price is fluid and can change at the discretion of the retailer or when availability or market demand changes. Additionally, a whisky may be expensive to one, fairly priced to another and relatively inexpensive to someone else. While the quality of the whiskey won’t change from year to year, the price most likely will increase from year to year.
I recommend scoring the whisky for what it is and let the consumer judge whether or not the price is fair.
Jeofother,
Thanks for dropping in and sharing your views. I appreciate the thoughts on personal budget future costly increases. As we state the price of the whisky at which the reviewer consider it, and state our views on that price hopefully those looking at the review in retrospect can extrapolate for themselves a value related to the price at which they find said whisky.
Jefother, thanks for the comment. Point well taken about individual budgets. Maybe another way to think about the price consideration is in terms of a comparison with other available options. To take an example: Macallan 18 Years Double Cask is $350 at my local, and Glenfarclas 17 Years is $100. Same region, stylistically similar, but I feel that Glenfarclas is not only a superior whisky in an absolute sense, but especially in consideration of the Macallan costing 3.5x as much. This site aims to be useful to consumers (who, like us, have to go out and pay for whisky with their own hard-earned cash) and the price sensitivity is an aspect of that which we’ll continue. Cheers!
The issue inevitably is that scoring is based on the reviewers own experiences and journey. There are plenty of bottles I’ve bought that Malt has scored 5/10 or 6/10 and I’ve been more than happy with. With previous reviewers, you learned their own style and whether you might actually rate that 6/10 higher (or lower) knowing their own love for (or hatred of) that style of whisky.
With the new influx of reviewers, the scoring system has been slightly scattered and there are plenty of whiskies that are receiving higher scores than I personally think they’re worth. Ultimately that means I put less emphasis on the scores by reviewers. Add to that the wholey bizarre world of tasting notes and I’m down to just caring about the preamble….The quality of which varies considerably.
Craig,
No doubt a bit of a tricky transition this year but hopefully you will begin to get to know us and find palates who more closely align with your own.
Graham
It’s not so much about finding someone with a palate close to mine, it’s more about trusting the palate of the new authors. And that takes time and demonstrating consistency.
If for example, I know you aren’t a fan of oloroso casks but give a good description and impression, I’m still inclined to look more in to that bottle, regardless of your score. But that relies on me trusting your judgement and understanding your style as an author.
Hi Graham
Really interesting piece, which definitely highlights a lot of my own thoughts and feelings when I joined Malt back in 2017 at the time the site moved on from being just Mark’s blog, which was when scores became a feature here.
This is probably a stupid question to ask a contributor, but I wonder whether you noticed the scoring bands here: https://malt-review.com/about/scoring-bands/
I think the descriptions attached to each score align pretty much with what you have in your table, though your addendum for reasons behind score increases/decreases is an excellent one – I think historically we just left it up to writer/taster discretion and expected there’d be an explanation where applied, but having it down in writing as you’ve done is very helpful I think.
Anyway, apologies for peeking back over the Malt fence, but your article made me think of those original bands and the reasons behind writing them and it occurred to me that perhaps they weren’t as prominently displayed as they could be, hidden in the “About” tab!
As an aside, just want to say how much I’ve been enjoying your pieces and all the articles written by new contributors in the last year. We went a good 3-4 years with mainly the same 4-5 authors for 95% of the content here and although I miss reading many of those writers it has been wonderful to read fresh takes and perspectives and I’m looking forward to more of the same in 2022.
Best wishes
Adam W.
Adam,
Thanks for taking the time to comment. Any yes I was aware of the bands but perhaps should have referenced them.
I know I am not venturing far from the scoring regime established by previous writers. But certainly did want to begin a more constructive discussion. Perhaps in future we could append the scoring grid or link back to it for easy reference from each individual review.
I also think others might have alternative ideas about these descriptors or the bands I’m open to explore other thoughts too.
Thanks for the post.
Personally, my main desire/request would be for scoring *not* to factor in value for money.
If reviewers simply rate on the basis of taste, then buyers can decide whether it’s worth paying, say, the extra £50 to buy a 7/10 bottle rather than a 6/10 bottle.
James, if I can jump in here: Malt is written by and aimed at whisky consumers. Part of the problem with professional reviewers is that they don’t pay for the samples they get, so they don’t ever feel the pain of an overpriced, underperforming dram. That’s great for them, but it means their experience is very different from that of your everyday whisky drinker. We aim to be maximally useful to the person standing in front of the store shelf, phone in hand, trying to decide whether to pay their own hard-earned money for a given bottle. If you see a positive (< 5/10) score from us, you can rest assured that a reviewer thought the whisky was good enough to shell out for themselves. I understand and respect that everyone has a different budget, but if you're the type of person who is so well off that they can buy any whisky regardless of price, you're probably drinking better drams than the majority of what we review, anyway. Hope this helps clarify our thinking on the matter? Cheers.
Thanks for the reply, Taylor.
I’m not sure if I’ve understood you correctly. You say, ‘We aim to be maximally useful to the person standing in front of the store shelf, phone in hand, trying to decide whether to pay their own hard-earned money for a given bottle’. Which is great. Often, however, when reviewers factor in value for money, it’s not very helpful. Or at least it doesn’t seem very helpful to me.
Suppose a reviewer gives a Deanston 12 a score of 7/10 because he thinks it’s great value for money, but gives a Deanston 18 a score of 6/10 because, although he prefers it, he doesn’t think it’s as good value for money. I don’t personally find that very useful to me when I’m trying to decide whether it’s worth shelling out the extra £30 for the 18 yo. (I’m certainly not the kind of guy for whom budget doesn’t matter.) I can’t even determine which whisky the reviewer would prefer to have if he was offered both free of charge. If, however, the reviewer simply reviews both whiskies on the basis of nose, palate, etc., and gives, say, the 12 yo a 6/10 and the 18 yo a 7/10, then I can decide whether I think it’s worth paying the extra, say, £30 for what’s only a slight improvement in quality.
I think you’re disagreeing with that, but can’t quite tell.
James.
James thanks for your comments.
As a whisky consumer I do find it difficult to score without thinking about what the bottle price means to me. Not everyone has the same budget as me, and to be honest my whisky budget varies throughout the year. But unless a tasting is truly blind it’s almost impossible to be swayed to some extent.
When it comes to scoring though I think about the price in relation to other products as much as in relation to any specific budget.
I think we tend to amend scores by exception. But hopefully by being complete transparent when we do you can deduce the unadulterated score.
Graham
Thanks for the reply, Graham. Yes, that’s true. If reviewers are explicit–e.g., ‘I knocked a point off for the price’–, then the unadulterated score can be backed out, which is helpful. Thanks, James.
The scoring scale is one of the things I like about Malt. I use the descriptions in the explanation page when scoring in my own notes (I have conversion factors for putting these on Whiskybase). I definitely think price and expectation are inextricably linked if it is whisky you yourself have paid for, and this should factor into a review, and mentioned in the text.
I have bought bottles on the strength of a 5 or 6 review on here. Ailsa Bay 1.2 Sweet Smoke springs to mind. Scored at 5, it was clear from the description that it’d be a hit with me as I’m a sucker for that kind of sweet meets peat whisky. That was indeed the case; my palate scores it a point higher. It has its flaws but at under £50 it met my needs and provided decent value.