Loyalty my arse

317261310_fc792058a9.jpg

Image courtesy of Simon Lord

Every morning as I meander to work in Charlotte Street I fortify myself for the day ahead at the Caffe Nero on Tottenham Court Road.

And every morning as I hand over the cash they parrot the same old question ‘do you have a loyalty card’. And every morning I mumble a ‘no’ and move onto the next question which is always about muffins or other items from the pastry selection.


Last week I ventured a more adventurous answer to their inanities, ‘you keep on making good coffee I stay loyal, you stop making good coffee I sod off’ (naturally this endeared me to the staff no end).
And that of course is the truth about loyalty.
Brand Loyalty is based on brand delivery, whether tangiable (great product performance) or intangiable (heaps of lovely identity value), it is not based on silly little loyalty card schemes or silly big loyalty card schemes for that matter. These are more accurately known as bribes and in a fairer world would be seen as a petty form of corruption. ‘Yeah I know that the product is shit, we are ripping you off and the service stinks but heres a bit of plastic that means for every half a million quid you spend in our store we will give you a half sucked polo mint and a bit of old string. Oh and occasionally we will send you a really poorly targeted bit of eCRM that will irritate the hell out of you but at least it won’t fill up your recycling bin like the other crap we send you. enjoy’.
OK, maybe I am a bit extreme as I refuse to take part in any reward programme as a matter of principle. But I am not content to simply pass up a free cup off coffee for every nine I buy, I really would like to see the whole stinking edifice of the loyalty business brought to its knees. For the simple reason that it perverts the behaviour of businesses.
For as long as organisations are focused on bribing their customers to continue purchasing from them they are not focused sufficiently on the real source of loyalty and that is providing a product that people find irresistible and refuse to substitute. And more than that, a product that people actively advocate in a slightly scary thousand-yard-stare kind of way.
Indeed I wonder how many brands or businesses that have these wonderful loyalty programmes actually have positive Net Promoter Scores – that’s where the number of customers that score the business 9 or 10 out of 10 for likelihood to recommend to a friend out-weigh the number that score the business 1 to 6 out of 10. With Net Promoter Scores you bin the 7s, and 8s as having a passive relationship with the brand and thus of no use to to man nor beast. Moreover, I wonder whether any of the brands that are really successful in creating strong advocacy have to bother with the bribes at all given their loyalty is based on far more fundamental relationships. Can you imagine an Apple Loyalty Programme?
So join me in my crusade against the business bribe, demand loyalty from brands rather than offering up yours for the price of a coffee and repeat after me ‘Loyalty my arse’, or as they might say in Cupertino if they weren’t so Californian and therefore allergic to bad language, ‘Loyalty my ass’.

Related Posts Plugin for WordPress, Blogger...

41 Replies to “Loyalty my arse”

  1. great argument.
    apparently the best performing Net Promoters in the UK throughout 04 were HSBC, ASDA, Honda and 02. I wonder whether they placed much emphasis on their loyalty scheme? Or whether they made good service an integral part of their business? I would guess the later would be true.

  2. Graham,
    Can I ask where you got your data from?
    I used to work in the banking sector and I don’t remember any of the banks doing very well, in fact I recall they mostly did terribly, bar a few exceptions.
    And I now work on Honda, so would be interested if that were the case (though not surprised).
    Ta,
    Pc.

  3. Interesting point – made more interesting because in my feeds only the top 2 paragraphs showed up and I still thought it was very interesting. The fact that every morning you go in there is an act of loyalty in itself. If the staff can’t be bothered to say hello, ask how you are and get to know that you’re in there every day then some would say they’re not worthy of your trade. How much nicer would it be if they remembered a face, and when you went in tomorrow they offered you a free coffee? Sure they’ve not put a stamp on a piece of card, but they know you’re in every morning, so why not? Surely training the staff to remember stuff like this would be cost effective versus a traditional marketing campaign where little pieces of card are stamped?
    As Iain points out here (http://www.crackunit.com/2008/03/25/make-mine-a-builders-superb-blogger-relations/) – nice and charming and personal are good things.

  4. I too generally refuse loyalty cards – for years I had a Homebase “spend and spend” card filling up my wallet – but never got around to actually spending the crappy coupons they sent out.
    I have just return to London after three years in San Francisco – and I was amazed when I went into the soup shop around the corner to buy my morning porriage on my first day and the guy behind the counter remember how I liked it! That really shows loyalty to the customer.
    Loyalty cards in the US are even more extreme. At Safeway they effectively put the price up if you don’t use your club card, so the loyaly card discounts look amazing until you do a price comparison. But the targetted coupons they provide based on your purchasing habits feed the American coupon culture.

  5. What’s your stand on airline miles programs? I hate loyalty schemes too, but I’ll never give up my *alliance Gold card.

  6. Hi Paul,
    The data is from a study done by the London School of Economics between 2003 and 2004, so the data isn’t that recent. The brands were judged against their sector, so although the banking sector might be terrible (the sector average was only -4.8) HSBC performed a little better than the rest in terms of NPS Vs Revenue growth. According to the data Honda performed well.
    I’ve got a copy of the report if you want me to fire it through.
    G

  7. Depends on your perspective, because on one hand a transactional loyalty scheme is a means of generating data into purchasing habits which can then lead to insights on how an offer can be bettered/improved, thus generating loyalty. And on the other, a revenue generating tool.
    Most of the current loyalty schemes out there are nothing more than elaborate tracking studies that reward consumer participation which is ok, if they’re recognised as just that!
    I tend to view loyalty as an an overriding/ongoing exercise that attempts to endear a brand with the customer and not the other way round.
    In this day and age, it is only customers who can reward or punish. Not brands.

  8. One free coffee every ten.
    That’s two free coffees a month.
    Assuming you’re full time, you’re passing up around £66 a year.
    In reality, you probably visit Nero because it’s on your route and because you’ve got into the habit of going there, not because they serve incredible coffee (they may do but would you really notice over another brand?)
    So in your case, loyalty isn’t even based on quality, just geography.
    Enter Game Theory, which says that you’re best off plonking your new coffee shop right next to the old coffee shop, so you’re immediately in with a shout of getting 50% of the customers.
    Actually, I find loyalty cards utterly tedious, too. But as it happens, I had a Nero’s one once and used it once to get my free cuppa in Soho Square. £66 says you feel just a tiny bit silly not taking them up on their offer.

  9. I see you are pretty sold on the intuitive simplicity of the Net Promoter Score approach to evaluating brand health. Is this just a strong opinion lightly held?
    The NPS is no panacea as I’m sure you know. The concept has been around for a while (cf. Jan Hofmeyr’s Conversion Model) and whilst its ability at predicting in-market performance is quite seductive it doesn’t tell the whole story.
    My (now ex) agency Leo Burnett have been measuring a nearly identical concept for many years which they called Momentum. It correlates quite well with in-market performance but they never use it in isolation because Momentum (and hence NPS) has some quite basic flaws if taken on its own …
    1) Whilst you may be willing to recommend a company to someone else based on past experience and perceptions you may not necessarily consider it to be ideal for you (even if you once did). A recommendation to another introduces an unspecified third party which makes interpretation a little tricky.
    2) Also, fluctuations in price can influence willingness to recommend when the product / service perception has in fact become poorer. If you’re not tracking price alongside NPS then you will miss this and the brand managers can “game” it without their bosses realising it.
    3) Competitor actions can of course often have a great impact on recommendation than your own. This may not be obvious to you if you are only staring at one number.
    Most importantly though a good brand health tracker does far more than evaluate how the brand is performing for use in some marketing dashboard — if you measure the right things with the right people then you can use it for diagnosis when things inevitably go south.
    I’m not saying that NPS isn’t a decent measure – it’s pretty good for what it is — but if you only measure one thing then you will only know one thing. If NPS starts going down rapidly you won’t have any data to tell you why.
    But hey, no one cares about quant in the UK these days so I’m probably talking to myself.

  10. I like NPS because it is so hardcore. You have to rate a company 9 or 10 out of 10 for your vote to count. Instinctively that works for the brands I love versus those I am merely satisfied with.
    I’m not recommending NPS as a panacea at all – just pointing out whether people would strongly recommend the brands whose loyalty programmes they belong to.
    If people are interested in NPS though there is a very good Mark Ritson study across a variety of secotrs in Australia and I may up load it if there are any takers.

  11. hmm…imho the key issue here is that these marketers are confusing loyalty (an attitude) with retention (a behavior). A loyalty card is no such thing. it is merely a retention tool, bogus bait so to speak, which will never connote active passion for a brand, but can only achieve a neutral, passive consumer reaction at best.
    yeah, us californians (even if some of us may have a german accent) may not know our ‘arse’ from our elbow, but that whole fake loyalty crap is pissing us off as well…
    retention, (pseudo-loyalty, as the great david lewis calls it), is dangerously misleading, because it can lull a brand owner into a fake sense of security.

  12. As a consumer, I too distrust and refuse to take part in sophisticated data-capture-dressed-up-as-loyalty-scheme initiatives like Tesco Clubcard, Nectar, Homebase Spend & Save etc. They feel insidious and a bit spooky.
    But I love the honest simplicity of Cafe Nero’s scheme. It’s a piece of cardboard. And a rubber stamp. They don’t capture my data, so they’re not going to spam me. I get more coffee as a reward, not something unrelated from a bad catalogue. And, because I like their coffee the best anyway, it feels like a nice, positive, rewarding thing for them to do, not the “mustn’t miss out” experience described by others.
    I agree that ‘loyalty scheme’ is an inaccurate term. But as a ‘frequency reward scheme’, I think Nero’s gets it pretty right.

  13. On NPS – I had assumed from this post and from your advocacy of NPS at the MRS conference that you had become a net promoter of NPS. My concern being that some of the more evangelical followers of NPS in the U.S. have thrown out the rest of their data in the misguided belief that they will know everything if they have that one number.
    On the loyalty card thing — I’m also a loyalty card abstainer. I put this down to a combination of privacy concerns and also to being sick to the back teeth of hearing how Tesco’s success apparently came from such deep insights as knowing that men buy beer when they buy nappies.
    I agree that loyalty cards should be considered as a form of promotion. And because they are permanent promotion (try stopping or devaluing a loyalty scheme) it can be argued that they permanently devalue the brand.
    However, there is the counter argument that consumers would put a higher value on getting their 11th coffee free than they would put on the equivalent discount off their daily cup of coffee. (A ‘behavioural’ economist will have done a study into that I’m sure – anyone know?) If there is also a retention benefit for Nero on top of that then it is a no brainer for them. Plus they’ve got your cash and you’ve just got an IOU so their balance sheet looks better.
    Another thought is that a well managed and appropriate loyalty card scheme *could* actually provide a mechanism for directly influencing staff behaviour and influencing true loyalty amongst customers: If staff rewards are linked to (audited) evidence of repeat custom then surely it could encourage the right kind of behaviour amongst staff? Or am I being naive?

  14. Lee,
    I brought up NPS at the MRS conference as a way of suggesting that the reason it was so popular in the boardroom is because of its (naive maybe) simplicity. Its what ever CEO has been crying out for – one metric that tells you everything. I’m not endorsing that view merely saying that that is the challenge research faces in the boardroom.

  15. Last week I received something in the post from ebay…It was a letter thanking me for being a ‘star ebayer’ (quite what I have done to deserve this accolade I don’t know) and a ‘£10 off my next purchase’ voucher as a way of saying ‘thank you’.
    That £10 off voucher probably wont change my behaviour towards ebay. I doubt I will use ebay any more or recommend it more frequently. However, that letter did make me feel a great deal of warmth towards the brand, and although my loyalty to them was never in question, I certainly would give the brand that elusive 9 out of 10 rather than 8 as i probably would have previously.
    If everyone that ebay sent the £10 to had a similar reaction to me (whether consciously or not) would it be money well spent? Would ebay have been better off improving the service rather than rewarding existing loyalty, as a device to engender further loyalty?

  16. Graham: I’d like to see the report too if you wouldn’t mind. Thanks.
    Glyn’s point about Cafe Nero’s loyalty scheme not being a data mining operation is a good one.
    But do people really mind if it’s money saved on what they buy anyway (E.g.: Tesco Clubcard or Nectar, after all, its not like shops don’t have any info on you anyway), or products they really like (E.g.: I have no issues whatsoever with Game knowing what games I buy in return for discounts and promotional mailings.)

  17. I don’t actually think consumers realise how much data is held on file and what it’s actually used for.
    If they did I’m sure they would be less inclined to sell themselves in such a frivolous manner.
    I for one refuse to hand over any personal details and do not participate in any loyalty schemes. And on a side issue, why are Tesco’s asking for and then providing data for TV licensing verification? They are traders, not the police.

  18. MM: I know they didn’t use to, but I thought nowadays it was common knowledge…
    Is it the case now that because people understand our data is everywhere anyway they are less bothered about giving it over in return for points/gifts/competitions etc?

  19. Rob: I completely agree.
    Consumers have become apathetic towards the ownership of personal data; they don’t want to know and they don’t care, because as you elude to, it’s all worth it for a freebie!
    Privacy is a commodity that is for sale. But then this is a bigger cultural issue shaped by politics.

  20. Right next time you’re in Tesco do what i do – give the club card points from your £150 shop to the person behind you in the queue. It helps bring fellow shoppers out of poverty and screws up the data guys at Tesco, creating a new niche group of customer who buy Moet, Innocent and Gu along with 19p (yes 19p!) Bubble bath and Tesco’s Value white bread.

  21. This is either one of those conspiracy theory, big brother is watching us type rumours, or a shocking an example of why these loyalty schemes are or could be used for more sinister purposes either now or in the future: A major supermarket cross sells life/health insurance. When taking a policy they do not only look at the information you have provided on the form, but your purchasing habits. So, someone who regularly buys cigarettes, lard and white lightning would be quoted considerably higher than a customer whose shopping list reads like Gillian Mackeith’s. Even worse your purchase history could be used against you if trying to make a claim. I just don’t like the thought that this could even possibly be true so have always avoided such schemes.
    I’d be interested if anyone knew if this practice was going on…

  22. I’d be interested whether Nero’s see the value in handing out points cards – maybe they do as they keep spending the money – to the coffee parents or the local office admin staff who do value the free cup or £66 /year.
    It’s always a problem when relatively affluent marketeers see themseleves as as the marketing activity target…….
    I guess Tesco’s see the value too – I think they just made £1.83bn profit have overtaken the other supermarkets since becoming a customer data centric business – faciliated of course by Clubcard data.
    I love research tools like NPS -never get bored of seeing how differently customers answer questions to how they actually behave as tracked through data, yes positive refferal is a ‘relationship’ goal – especially if we agree relationships are with ‘brands’ not programmes – even if facilitated by a piece of plastic – that also happens to provide a direct to customer brand commuication channel which is acutally quite important for brand marketeers in the days of media fragmentation.
    Just thoughts as a loyalty markeeter bored of the ‘anal’ discussion on the value of loyalty by brand marketeers who think they understand a different subject – still I am interested in brand relationships so watch this space – maybe loyalty programmes can start taking on previously defined brand marketing tasks…

  23. I can’t be arsed with coffee shop loyalty cards. For similar reasons to you, Richard. Much more powerful when – as has happened to me once or twice – the manager of the coffee shop just says, ‘You come in here every day, don’t you? This one’s on us.’ That way I feel special, not just part of a scheme.

  24. The problem here is surely a categorization error. This is not a ‘loyalty’ scheme. It’s sales promotion; buy ten get one free.
    Real loyalty is more important than ever for brands, but that doesn’t necessarily mean cards, points and prizes.
    According to a recent Henley Centre Report I read, affluent consumers now want intangible benefits from their brands and, interestingly, relevant information and expertise. If brands can find a way of providing that, loyalty will surely follow.
    It might go well beyond the traditional cards and points and rewards. I like the Orange phone tent idea at Glastonbury, where they provide free mobile phone charge-up facilities – useful, practical but also touchy-feely and warm.
    What one person wants from the brand might be very different from what another person wants. So the trick is to find out what they want and give it to them – or better yet ‘surprise and delight’ them by anticipating what they want before they know it themselves, which is something we’ve been working on with a client of ours recently.
    But having said that, you can’t underestimate the power of ‘free’. I always remember being so annoyed when The Times ran its 10p a day promotions and I couldn’t get a copy to do the crossword on the way to work. No other paper would do for me, because of the crossword, but clearly some people just don’t care. They’ll switch their brand to save about 30p (which was the difference at the time). I found it unbelievably pathetic that anyone would do that; indeed I even thought about offering someone 30p for their copy of the Times so they could buy the paper they really wanted to read and I could have my crossword.
    You would think people have a real ‘relationship’ with a paper surely; the journalists, the writing style, the political stance, the layout, the sections etc. I hate reading unfamiliar papers; I find it disorienting. But it seems a saving of 30p is enough to end this intimate relationship for lots of people. I find that a bit sad and cheap myself.
    I imagine a saving of £66 a year on coffee is pretty fantastic to a lot of people.

  25. Is it just me (working in the provinces) or does spending £660 a year on a morning coffee, that by all accounts you don’t seem to appreciate massively, seems like a COLOSSAL WASTE OF MONEY?
    I’m assuming in your significant offices you have perfectly good coffee-making facilities?

  26. A lot of talk about loyalty, promotions, and other marketing nonsense, to me the Nero card is just a simple way for them to say thank you and offer a bit more value. Does it increase your loyalty / frequency, probably not, is it a mechanic that acts as a surrogate for the coffee shop owner giving you a cup on the house – no, it’s just a nice gesture.
    I think we forget that we have nice jobs, where as the people behind the counter at Nero have to look at our caffeine deprived contorted faces as they run through the script of “do you have a loyalty card” “do you want any muffins or pasties with that” as we grunt at them.
    But the Nero card does do one really important thing, it gives the stamper power, the power to give you two, three or as i have experienced on many occasions however many stamps i need to fill my card. On these occasions that simple mechanic has left me with way more than just a hot coffee.
    So the next time you’re fiddling with your blackberry, making a call or just cross because the queue is more than 5 deep, be nice to the people behind the bar, give then your full attention when ordering, smile and see what happens.
    The point here is that front line staff generally have no authority to action anything, what the card does is give them an element of power which they use at their discretion. These acts of kindness really do create advocacy.

  27. Rewards don’t create ‘loyalty’ but they can create additional positive equity around the brand at the subconcious level – some of which transforms itself into future purchasing intent but, only if the product is any good.
    So Richard – ‘Loyalty my arse’ indeed.

  28. Hmmm… feels to me like everyone posting or commenting here doesn’t usually clip coupons out of newspapers to save money on their weekly shop. Many people do though, and I imagine they are who the hugely successful Clubcard scheme is aimed at.
    Clubcard is, I understand, one of the few loyalty schemes that actually works. And Tesco clearly outmanoeuvres the competition partly based on the deep customer insights from analysing Clubcard data.
    Also, many people probably swipe their loyalty cards out of habit if nothing else – collecting points is a basic gaming mechanic, not so different to collecting friends on Facebook.
    I absolutely agree with the central idea here that brands need to earn loyalty with every encounter rather than using bribes. But that doesn’t mean that price/discounts/offers can’t still play their part for the large number of lower-income consumers who probably don’t spend time debating the value of Nero loyalty cards on marketing blogs ;-)

  29. Andy – nice one – point though – offering discounts to lower-income consumers makes them addicted to discounts – it doesn’t however constitute loyalty.

  30. Loyalty my arse – good point, I think you mean Arse Loyalty?
    I spend a bloody fortune in Sainsburys and still don’t have a nectar card but do feel good giving my school vouchers to the lady behind me at the checkout (and Jesus loves me for it).
    Arse loyalty is all about behavioural loyalty, bums through doors darling. Mainly done through store location (convinience) or some crappy sales promotion like stamping another thing to cram in your wallet – normally because some competitor has also twigged that being in a high regular footfall location may be a good idea.
    Your scores refer to Heart or Head Loyalty which is very different. Us planning chaps love this but actually the end goal is still arse loyalty which is what turns a marketeer on.
    So, I am heart loyal to stuff I really love obviously but then this may be far removed form sales. Heinz salad cream a great example – everyone loves it, its a british institution (in thi s case nostalgia) but no one buys ut until its threatened to be taken away form them.
    Head loyal, stuff I have a more tangible reason for, like listening to radio 4 podcasts.
    Anyway, it would be interesting to classift different motivations or influencers for these types of loyalty and see where the best connection points to leverage are and even profile people with it?
    E.g. love of giveing coupons to random housewives + love of salad cream and radio 4 = sinister and strangely dull planner, we’ll call this target spanners? Touchpoints? best not to ask.

  31. Hey, would it be wrong if my interest in the article was based more on coffee than advertising? Either way…
    I go to the same coffee shop every day and get an espresso after lunch for 1 euro (!), cheapest in Dublin. It’s run by a couple who I’ve got to know over the last year or so. They know me, we call each other by first name, their espresso is one of the best if not the best in Dublin.
    I’ve gone full circle from recommending them to everybody, to kind of guarding them as my secret and telling nobody.
    So I’m getting value for money, service, quality product and no inane emails / cards to stamp.
    And no, I’m not telling any of you where it is ;-)

  32. Asking that you stand there patiently and fumble out a piece of loyal plastic long forgotten in your wallet; they then expect you to shunt off and move aside, the moment they hand you back your card, a note, several coins, a receipt you’ll never read, and a voucher rewarding your loyalty. If you’re lucky you remember to pick up the coffee too, and if there’s enough room, time, and a hand spare, you’ll get to add the milk, sugar (stir), cocoa powder, and finally the lid; yourself. And exit, burning your tongue, dropping the loyalty card on your way out.
    Bad start to the morning.

  33. Good provocative post.
    Been arguing much the same on the imbalance of “loyalty” as marketing normally construes it for some time (not least in the Pink Book), no matter how many CRMers shout and scream…
    Ehrenberg has always been good on this, pointing out that loyalty is both highly overvalued and largely a function of other stuff so therefore not a very good thing to base a game on but the latest academic recruit is the otherwise debatable Gerry Zaltman http://hbswk.hbs.edu/item/5871.html (ignore the deep-metaphors-shape-individual-behaviour nonsense and go straight to the extract from his latest book Marketing Metaphoria)
    For my money the big question is why the idea of consumer loyalty so appeals to marketing folk and what does it reveal about our other ideas, given it is, as the man say “Arse” (and not in a good way)?
    BTW congrats on the anniversary! Still one of the best blogs in the space to get the leetle grey cells moving…

  34. So I thought your Loyalty post was interesting. For a while, I didn’t believe in loyalty cards either – with the exception of airlines, which for the sake of transparency I’ll mention. For me, though, my Damascene moment was down to Michael Moore – at his Roundhouse show several years ago, he questioned the concept as well. I’m paraphrasing here: what does it say, he asked, if your loyalties lie with a supermarket? Shouldn’t your loyalties be to your dreams? Your hopes? Your ideas?
    Obviously somewhat mawkish, and I certainly don’t want to suggest Michael Moore is anything like a saint – but it made sense to me. And to many others, all of whom fished out their loyalty cards from their wallets and took them up on stage to be subject to a little public brand execution. And I felt a little better about myself afterwards – unencumbered. I still do, in fact – every time I go to Sainsburys and they ask me at the till, do you have a Nectar card. I say no, and I feel good.
    Several people have pointed out to me that I’m being pointlessly stubborn – after all, I do use Sainsburys on a regular basis. What about the individuals for whom it makes a difference? This still troubles me, though again, like someone else above, I try and offer my points to people in the line behind me.
    I guess the main point for me is that Sainsburys says nothing about my dreams, hopes and ideas, and so I wouldn’t associate my brand – me – with theirs. I’m more happy to ally myself with Virgin Atlantic, on the other hand, and not just because the savings are potentially greater – if I were to make a list of brands my brand would do business with, they’d be on it. I’m not so interested in, or worried about the data capture aspect – I know well enough to tick, or not tick the box that in theory excuses me from mailings. I’m more worried about what that loyalty card says about me, if someone glanced in my wallet.
    I wrote a story a while back about a couple who, on a second date, play a game with their wallets – wallet trumps, the game was called. They sit there in a cafe and go through all the cards in their wallet one by one, trying to outdo each other as they place each card down. It wasn’t a particularly good story, but I think the point is, if I were playing that game, I’d now happily argue why my Coffee Republic Card beats out a Tesco Club Card.

  35. I can’t be bothered with reward cards either – but I too am an over privileged agency type . In truth Nectar, Clubcard and the Cafe Nero Card are highly successful and hugely popular, millions of consumers willingly participate. I expect Nero has a positive net promoter score, I’m less sure about Tesco and Sainsbury.
    You rightly make a distinction between retention and loyalty. And you’re absolutely right to say loyalty comes from brand and product delivery. Yes mis-targetted comms are irritating. But you’re absolutely wrong to assume that the people don’t appreciate their 10th cup free or money off their next weeks shop especially from a brand that recognises Every Little Helps.

Comments are closed.