Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
Role Playing (Games)

Designing a MMORPG Feedback System 86

Posted by Zonk from the yes-please dept.
Gamasutra is running one of their highly enjoyable 'soapbox' pieces, looking at possible ways you could implement a feedback or ranking system for Massively Multiplayer Online Game players. From the article: "When playing an MMORPG, I should be able to give a positive, neutral, or negative rating to anyone who has been in my group for more than thirty minutes. Negative ratings could be characterized via a multiple-choice list of common gripes (i.e. 'loot theft', 'abusive language', etc) -- a feature now built into the Xbox Live feedback system. However, it isn't clear that a good feedback system requires this level of depth; there's an argument to be made for simplifying the process as much as possible."
This discussion has been archived. No new comments can be posted.

Designing a MMORPG Feedback System More

Designing a MMORPG Feedback System

Comments Filter:

  • Sliding Scale (Score:4, Interesting)

    by biocute ( 936687 ) on Thursday February 02, 2006 @04:59PM (#14629631)
    Just a Slashdot-style moderation system, where a player can be modded up and down (one vote per person, which can be up/neutral/down) according to that player's action.

    These mods will "expire" after a couple of days so that players can start a new leaf.

  • decent idea... (Score:4, Informative)

    by HTL2001 ( 836298 ) on Thursday February 02, 2006 @05:00PM (#14629637)
    well the problem with most rating systems used is that they could be abused... this sounds good though, since you would need to be in a party for a while...

    FYI - this was tried in Ragnarok Online but it was taken off due to abuse. People would pay (in game) money for positive points, and you'd have people who would stalk you and give negatives at every opertunity. You could only give 1 point per day, but it was still abused by people having multiple characters on one account or multiple accounts. It opened people up to a lot of threats as well, as a planned implementation of the system would have been more chance to drop equipment when you died.
    • don't underestimate peoples desire to slander off others. If someone has multiple characters it's easy to relog with a character your victim doesn't know, join their party, spend the prerequisite time with them and then give them a black mark. Then rinse, wash and repeat. One thing I've seen great evidence of is that when people want to they can be extremely malicious which is why I think a system like that, at least in an MMORPG would not work.
    • Hate to reply to myself, but there's one more thing.

      Retaliation.

      If you're out in a field leveling, you know who just marked you down for dumping something on you/kill stealing/etc.
      While your negative point to that person is justified, that person can just fire back at you with a negative. Roughly the equivalent (though not as severe) of telling a criminal who turned you in and where you live... it wouldn't exactly encourage you to use that end of the system.
    • Agreed. When it comes to online games, people become very competitive and opportunistic. In almost any MMORPG, I'd say implementing a feature like this would result in the following:

      1. Players paying in-game or real life money for positive points
      2. Griefers who throw around negative ratings for the hell of it
      3. Large guilds / clans that will dilute the meaning of the ratings by boosting their members up while boosting rival members down
      4. Large guilds / clans gaining way too much influence in the game as
    • I play Maple Story (www.mapleglobal.com/ ,sadly site aka the download of the game is only IE supported) and they do this with Fame. Later on in the game however you need fame to do certain quests and wear certain armour.

      Pros of this system: You have to wait till lvl 15 to fame or defame. Meaning a person couldn't easily create multiple characters just to fame or defame someone. Additionally you can only have 3 characters on any given server. So even if they did get them all up to lvl you could only do it
  • "I understand concerns that a reputation system might drive away paying customers who get bad ratings..."

    Has bad Karma ever driven a Slashdotter away?? These types of community style initiatives usually work out pretty well and generally add to value to the experience but I don't think it would drive me over the pay/don't pay edge. :o)

    • that's not so. Think about the difference between /. and a MMORPG. If a guy has bad karma on /. then so what, it's just a forum and it's a place for chucking mud and stick and stones; However if a guy has a bad rating in an MMORPG then it becomes impossible to group, whether the rating is deserved or not. So if people find a way to manipulate ratings and attack someone in that manner then it ends up detracting massively from the victims playing experience.
    • Slashdot karma has no particular affect on my enjoyment of the website.

      If I'm playing a game, and I have a red flashing light over my head that says "LOOT THIEF", that, however, will impact my opportunities within the game, and my enjoyment of it -- especially if I'm paying money for the privilege.

      The problem is that no part of a game 'regulates' gameplay, as if from heaven -- it only changes the nature of the game being played. Lots of people will be delighted to discover the "apportioning blame" sub-game

  • I doubt it could work. (Score:4, Insightful)

    by JoeD ( 12073 ) on Thursday February 02, 2006 @05:02PM (#14629664) Homepage
    Any such player-run system would be too easy to abuse. If you put restrictions in place to prevent the abuse, people will game their way around them.

    For example, a guild could have all of its members give each other high ratings. Or they could band together to give poor ratings to someone that pisses them off.

    • The key, then is to not give out absolute ratings, but rather relative ones. The rating you personally see for people you haven't rated yourself should be based on how they're rated by people you yourself have rated highly. This goes a long way towards preventing abuse by effectively negating the ratings assigned by people you don't like.
    • No one said that making a non-abusable feedback system was easy. :)

      One easy way to prevent the "mob mentality" is to check to see what affiliations/connections people have when they rate a person. If I'm an asshole, I'm probably going to be an asshole to everyone, NOT just one guild. So theoretically, I should be getting negative feedback from a wide, diverse group of people. The same applies if I'm a nice guy.

      So, if a guild tries to work together to boost each other's ratings, or tank someone else's, the f
      • Has any MMOG made any system that can't be abused?
        • Not that I'm aware of, or probably more likely ... the MMOG isn't popular enough. There are plenty of games that don't have the griefer problem that's prevalent in WoW. By many personal accounts, the EQ2 playerbase is more "mature", probably since it's no where as popular as WoW. Then again, when I was playing the first EQ and THAT was the big hit game, there were plenty of griefers there too. Same with UO when it first released.

          Arguably there is feedback/moderation system that can't be abused. We all know

  • Didn't we just discuss this? (Score:3, Interesting)

    by AKAImBatman ( 238306 ) <akaimbatman AT gmail DOT com> on Thursday February 02, 2006 @05:03PM (#14629685) Homepage Journal
    Now that you mention it, I do believe we did [slashdot.org].

    The result of the conversation was that a rank system could be implemented similar to that of the Military ranks. Rather than working your way up some sort of point system, you'd get your promotions directly from your CO. (That's Commanding Officer for those of you who aren't familiar with the term.) High ranks would be seeded by the "Admiralty" (i.e. The people running the game), thus kicking things off. And if you really don't want to follow the whole rank system, you can always privateer.

    At least, it seems to work in the context of Star Trek and military SciFi. YMMV elsewhere. :-)

  • "feedback" in RPG's out of character? (Score:3, Insightful)

    by alkaloids ( 739233 ) on Thursday February 02, 2006 @05:04PM (#14629693)
    I play Gemstone, an older (to put it lightly) text-based MMORPG. In it, someone's "popularity" or whatever comes from interacting with other characters etc. Having a game mechanic to tell you if someone was a jerk to someone else seems very very strange, and would definetly be out of character. So, for "role playing" games I don't feel like that is something that's useful or good. On the other hand, if the game has on objective, and you are trying to "win" WoW or whatever, I guess having that feedback system could save you time and frustration. I guess one way to rationalize this in character is to have some little cottage where you would go in and greet a jolly fat man in a little red suit who keeps track of who's "naughty" or "nice".
    • I play Gemstone, an older (to put it lightly) text-based MMORPG.

      Dude. That's a MUD, not an MMORPG. I know the basic concept is the same, but the two *are* considered to be quite different.
    • As a veteran of GemStone (11 years at this point, I think) I can say that game is much more about cliques and social circles for your "standing." MMORPGs are really only MMOGs for the most part, with some exceptions here and there by outstanding players. In those games, having a ranking system only slightly rationalized (or not at all) would be an improvement regardless, at least when you're dealing with pick-up groups for raids and you're dealing with total strangers.
    • having another Reputation grind in WoW is definately not what we need. Especially with the kind of elitism and backstabbing that goes on. Also there is another element to this, which is that a rating system is only useful if you don't know someone. On the average WoW server where the population is well established most people know each other or or at least know peoples reputations. This is because it's an immersive world rather than a a single purpose based game like Quake. In either case we don't need ebay
  • You should use a distributed trust metric [slashdot.org] system like Advogato [advogato.org]. This allows you to develop a hierarchical system for rating peers in a community.
  • "/target: age?"

    And I'm good to go.
  • An excellent idea
  • Yes its a way to keep track of the people who play but also it builds up a stigma that a big red X shall be assigned to me by anyone who is unhappy with my performance... and what is the life cycle of such an X and the what if the good fairy comes by and gives you a + for whatever reason, which do I believe and then what about the entire lables thing.. Isn't that what we are trying to get away from in the digital realms? Away from lables and classifications... man sounds like school and citizenship. Class c
    • Try playing any online game (not just MMORPGs). I guarantee that it won't take long before you run into asshats that ruin your game experience in some way.

      Is it a bad label? A stigma of some sorts? Absolutely! But that's what you get for being the jerk, the griefer, etc.

      As for "class clowns", there are ways to be entertaining without ruining the game experience for people. Such a feedback system would, hopefully, separate those that we don't mind seeing (the silly "class clown" types) versus the asshole who

  • Won't work (Score:4, Insightful)

    by dc29A ( 636871 ) on Thursday February 02, 2006 @05:08PM (#14629740)
    I should be able to give a positive, neutral, or negative rating to anyone who has been in my group for more than thirty minutes

    Hypothetical situation. I form a group, recruit people who want to do (insert dungeon here). Halfway through the run I realize one of the member is incompetent. I remove him and flag him "incompetent". Out of spite he returns the favor and flags me something negative. Yet I've only done what was best for the success of the run, I wasn't using abusive language, I didn't do anything wrong.

    Another problem is when there are games where you can reach max level fast. Blacklist the assholes all you want, they will remake as different characters. UO murderer system is perfect example where griefers would level throwaway characters and PK until they got caught. Rinse, repeat.

    Another problem is with raid guilds. From experience, raid guilds will recruit the biggest assholes on the server under two condition: They obey the raid leaders and have insanely high play time. Raid guilds don't care if you are an asshole. They care only if you can contribute to their success.
    • Hypothetical situation. I form a group, recruit people who want to do (insert dungeon here). Halfway through the run I realize one of the member is incompetent. I remove him and flag him "incompetent". Out of spite he returns the favor and flags me something negative. Yet I've only done what was best for the success of the run, I wasn't using abusive language, I didn't do anything wrong.

      Your entire group gives him bad feedback. He gives the entire group bad feedback. You each have 1 bad feedback. He has
      • Your entire group gives him bad feedback. He gives the entire group bad feedback.

        ...and then the twit whom you kicked from the group not only gives you bad feedback, he logs on with his brother's account to give you bad feedback, then runs off to his favourite forum to explain in detail how you kicked him from your group because you wanted to get all the loot for yourself and he had rolled higher than you, and begs everyone he knows to slap you with bad feedback too...

        And it just goes on from there.

        Th

        • Consider then how you determine what you believe about someone in the real world. I imagine it's a function of many different pieces of information you know about the individual all rolled up via some crazy black box methodology (your brain) into some sort of personal judgement. Psychologists have done quite a bit of work on how people categorize others- esp. with respect to the "In my tribe" vs. "Not in my tribe." The question is not "can we do it," but rather what's the benefit and what's the cost? Th
        • Well, one of the ways to avoid such things is to use systems like the article mentions, or like Xbox Live has - you can't just give feedback on anyone out there. The article limits it to someone who's been in your group for X amount of time, and XBL requires you to have played against someone online. This all the begging and pleading of some idiot to his friends to give you bad feedback isn't going to work, since they aren't allowed to do it just because they know your name.

          If they have a good enough memo
        • A safegard against this kind of behaviour is easy. Set it to where they have to be grouped with you for a period of say, 15 rl minutes, before they can give feedback. Since most raids take MUCH longer, it could be a deturrent. You can also set a threshold to 1 rating per day, per account. In other words, you may not leave more than 1 feedback for a particular account, on a particular account. On top of that, when someone is kicked from a raid/group/party/whatever, the leader may then be presented with the o
    • If that's the case, and someone is only participating in raids, then they should have a high community rating, shouldn't they? I mean, I don't really want someone critiquing the way my language "sings" on Slashdot - I write to make a point, and then submit. If I'm writing poetry, I do it on a different system (like Everything2.com) and get rated on my poetry there.

      MMORPGs aren't just about your goals. They're about shared goals.

      If someone wants to be an asshole, but can curb his anti-social behavior to p

  • Violation specific feedback (Score:3, Interesting)

    by Otonotachibana ( 826415 ) on Thursday February 02, 2006 @05:09PM (#14629742)
    One of my gripes with WoW is the inability to quickly flag someone as a gold-seller, user of inappropriate language (sexist, homophobic, racist, etc.), exploiters, etc.

    Too often will I receive a whisper from a character trying have me visit a gold for $ site. I will report them but the process is not stream-lined, it takes a few minutes. If multiple people could quickly flag this character as a gold-seller then blizzard reps would be able to investigate and perma-ban the most abusive accounts.
    • The easier they make it the more abuse it will have. If someone just click a button and it reports then abuse will go up. When someone has to fill out forms for 2 minutes then abuse gets cut down because people don't want to waste all that time.
      • I agree with your comment. Reports of abuse would go up with a click-to-report system. The benefits would be that a blizzard database could rank the most-abusive individuals. Blizzard reps would take a look at the suspected individual's whispers. If there was specific evidence of abuse the individual would be subject to punishment per blizzard's rules. This system would target the most damaging members of the WoW community rather than have the blizzard reps read through page after page of GM tickets (w
    • the problem stems though from the poor game masters that blizzard employs and their policy towards gold selling. If a certain character was flagged by dozens of people as a gold-seller it would make fuck all difference to a GM because it's simply not important to them. they don't care. I say this because every single thing that is said, whether /tells or in general chat is logged. If it was in Blizzs interest to stop gold sellers they would. They simply don't have the capacity to stop the gold sellers eithe
    • The problem is griefing someone unfairly in such a manner. A company like Blizzard doesn't have the manpower to resolve everything on the spot when it happens. It could work for smaller games, possibly, but even then it seems risky... you put checks here and balances there and eventually there's so much back and forth you might as well not bother implementing any of it.
  • Especially the fucking rule about abusive language. It's lame.
    • You must be one of the players who characterizes anything they do not like by calling it "gay".
      • No.

        I just dont like rules about what I can write and what I cannot. Ofcourse, I should choose my friends accordingly, but I really dislike hypocrites that think it's better to kill civilians in iraq than to use a word.
  • it's called /yell in Ironforge. Seriously though, A MMORPG is such because it's an immersive world where people not only quest for 'phat lewtz' but spend a good amount of time sitting around, idling and chatting. I don't need a feedback system to tell me that X is a ninja or that Y is an excellent healer or tank that I would be happy to have in my party, I know from experience and communication. Jeez it almost sounds like handing out gold stars.

    Plus a system like that is open to massive abuse, especially wh
    • it's called /yell in Ironforge. Seriously though, A MMORPG is such because it's an immersive world where people not only quest for 'phat lewtz' but spend a good amount of time sitting around, idling and chatting. I don't need a feedback system to tell me that X is a ninja or that Y is an excellent healer or tank that I would be happy to have in my party, I know from experience and communication. Jeez it almost sounds like handing out gold stars.

      The problem is that AFTER THE FACT, you will find out that X is
      • I've been playing on my WoW server for a year now with no transfers so the community is well established. Obviously PUGs aren't that common now as most people are guilded but in the instances where I don't do guild runs the PUGS I do are hand picked people I know are good at their roles. This comes about through knowing the community I'm part of. Any ninja's on the server are pretty well known and are pretty much avoided; If some poor PUG is unlucky enough to pick up a ninja then what they learn from that e
        • Well see, that's the thing. You play in a community that's been well-established (no transfers). Not every server is like that. You also mention that, since you know the community, you know who to pick when doing PUGs, etc. It's very much a rite of passage, and one that I can understand, but that doesn't make it any better to the victim of a ninja or griefer. Not everyone has the benefits that you do. Plus, the goal is to come up with a system such that even new users can benefit from.

          I'm not trying to argu
          • there is definately room for improvement, in that I agree with you. However I feel that the improvements I'd like to see the most are not technological but social, including how social interaction takes place in these worlds and the importance of players moderating theirselves more closely. What spoils any MMORPG the most for me is the childish and inane chatter in general. In that respect I would be very happy to see an MMO with a minimum age requirement. Like I've said before, I think any rating system w
            • there is definately room for improvement, in that I agree with you. However I feel that the improvements I'd like to see the most are not technological but social, including how social interaction takes place in these worlds and the importance of players moderating theirselves more closely. What spoils any MMORPG the most for me is the childish and inane chatter in general. In that respect I would be very happy to see an MMO with a minimum age requirement.

              I've very much for reducing stupid-talk in General

  • Because it worked so well for ebay (Score:3, Funny)

    by StikyPad ( 445176 ) on Thursday February 02, 2006 @05:23PM (#14629885) Homepage
    "A++++++++++, great raider. Would group with him again."

    "Perfect, now hand over the Scimitar of the Wolf, or my whole guild will give you retaliatory feedback."
  • Any company that is able to guarantee a favorable experience in an online game is going to make a lot of money. It's not just MMORPGs that need this, but FPS titles as well. How many times have you played a team-based FPS (like Counter-Strike), only to run into some asshole who thinks that shooting you in the back constantly is "funny"? Many games have tried changing the game mechanic to prevent this from happening (for example, if you cause enough team kills, you get booted from the server), but a lot of t

    • How many times have you played a team-based FPS (like Counter-Strike), only to run into some asshole who thinks that shooting you in the back constantly is "funny"?

      In my quake/tribes days I used to get booted and banned from servers because they thought I was using bots. Your vision isn't flawless.

      • Are you trolling or something? Obviously any feedback/rating system has to take into account things like false positives (maybe the guy really IS that good) or revenge scenarios (griefers trying to abuse feedback systems themselves).

        You probably got booted from the server because some jerk admin thought you were using a bot, even though you weren't. But if you put some software in front that does the detection, a la Punkbuster, then you don't have to worry about things like jerk admins. If you still get boo

  • Fable anyone? (Score:2, Interesting)

    by The Rabid Panda ( 951579 )
    What about a system like Fable had? Not necessarily changing your appearance, but your fame (or infamy) was automatically calculated by the game, and NPC's react differently based on your deeds. Say the system is based on points; playing all the way through cooperatively in a quest gave so many positive, healing teammates, etc. On the flip-side, ninjalooting etc. would be caught by the game mechanic and weighed accordingly. This "karma" system would be automatic and players could say, view a few stats abo
    • If activities like ninjalooting could be caught automatically by the game, then why not simply disallow them?

      All that that kind of system would do is encourage players to rip off their teammates, knowing that it is allowed by the game and that it only costs them a handful of "dishonour" points which can be easily worked off by a couple hours of two-boxing and healing your other character.

      • Because the ultimate idea of any MMORPG is immersion. Limiting the ability of any player to do as they will (within reason) will make the game less desirable.

        As for working off their rep, that could be handled in such a way that the cost/benefit is either dead even or slightly against really undesirable acts. Then they would be MUCH less likely to loot, but the possibility still exists.

        People will alway loot, cheat and PK. BUT, if it becomes a lot less desirable to do so just because it was easy, I
        • Richard Garriot once told a story about the early days of Ultima Online. I can't find the original source, but it went something like this.

          He was wandering around the world as Lord British when he saw someone robbing another character. He promptly used his godlike powers to freeze the thief, demanded that he return what he had taken and gave him a lecture about not stealing. As soon as he let him go, the thief stole something again and ran off.

          When he confronted the thief over why he had continued to

  • Identity verification (Score:3, Insightful)

    by Ruprecht the Monkeyb ( 680597 ) * on Thursday February 02, 2006 @05:36PM (#14630026)
    None of this will matter until there are real consequences to anti-social behavior. That requires a couple things. First, you need the people running the game to care enough to *publicly* boot people for being asshats on a regular basis. Secondly, once that is done, you need a way to prevent the asshat from just opening another account or creating another character.

    I've thought for a long time what the MM market needed was a third-party identity broker. Call it 'Good Guys, Inc'. You apply for a GGI account. GGI does a check, verifies you are who you say you are, address, SSN, whatever, and issues you a GGI #.

    You buy Worlds of Evercrack, and enter the CD key that came with the game into your GGI account. You login to a GGI-enabled game server, and it uses the gamekey to lookup your GGI account and verifies that that key belongs to a member in good standing. Or, if you've been banned (from that game, or anther game, or whatever they want to check), it boots you or forces you to play on a server comprised of other asshats.

    Not infallible, but maybe if it booted the jerks (and ninja-looters, and bots, and pro farmers) from not only that game, but potentially other games that used the service as well, things might start to improve. Or at least segregate the people who want to play the game from those that want to beat up on other players.
    • The obvious problem is one of accountability.

      "Hi, Welcome to the Asshattery server, a GGI-positive shart of the Generiquest game. Since you're new here, here are a few rules:

      "First, don't be a dick. You'd think that you would know that, but sometimes we have to explain things.

      "Second, look for anyone with the <Uberguild> tag over their heads. These people are your gods. Not doing exactly what they say, be it leaving your current campsite, loaning them all the gold in your pockets without ask

    • None of this will matter until there are real consequences to anti-social behavior. That requires a couple things. First, you need the people running the game to care enough to *publicly* boot people for being asshats on a regular basis.

      So you favour a game in which the characters are not allowed to behave like real people?
  • Feedback systems can always be exploited in some way. Nobody would want to give feedback, even if the a$$hole in the group deserves it, in fear of getting negative feedback in return. You'd have every player ask for positive feedback after every dungeon run, which would be annoying, and it would be hard to say "you did alright, but not quite good enough to deserve a positive rating" to mediocre players you got along, but didn't really have a great time, with. Honestly evaluating people may offend them.

    eBay'
    • Feedback systems can always be exploited in some way. Nobody would want to give feedback, even if the a$$hole in the group deserves it, in fear of getting negative feedback in return. You'd have every player ask for positive feedback after every dungeon run, which would be annoying, and it would be hard to say "you did alright, but not quite good enough to deserve a positive rating" to mediocre players you got along, but didn't really have a great time, with. Honestly evaluating people may offend them.

      Would
  • Just in MMORPGs? I think we need to begin taking more actions to rate people we interact with in everyday life.

  • Eve Online (Score:2, Informative)

    by Odin_Tiger ( 585113 )
    Once again, I brink up Eve Online on a subject under games...I swear I don't work for them, tho. :P
    They have two great means of this. 1, players can set standings for how they view any player, corporation (think guild), alliance, faction, or what have you. The can rate some(one/thing) on a scale of -10 to +10. At any time you can look at another character's standings to see what PC's, NPC's, Corps, factions, alliances, etc., like or dislike them. Also, committing illegal acts in secure / semi-secure spa
    • I'd have to agree that Eve Online takes more of the correct approach to ranking/feedback. How you play the game should generate consequences from the game. Ranking/feedback needs to be part of the gameplay not an out-of-character survey.

      If you piss people off repeatedly then it should show on your character somehow. You have lots of scars and a broken nose. Your ship is all beat-up and shoddy looking showing that merchants don't like you.
  • Basically, different people think different things are good and/or bad in another person's playstyle.

    Examples:

    You are a person that likes to take their time going through a dungeon/instance/raid. You want to be thorough. However, you aren't able to find a group with your regular friends, so you do (oh, the horrors!) a PUG. The people you get stuck with are of the OMGZERS HURRY UP! mode. You stay with them, because you don't want to get a horrible score for leaving too early. Your play styles clash too
  • Wasn't this tried in The Sims Online, where it resulted in the community being run by a mafia of players coercing people into following their orders through threats of negative feedback?
  • I've been running an online game for five years now (see .sig). During that time I've experimented with several feedback systems.

    None of them work as they should. The problem is that there is too much abuse in the system. People who want to game a system will always find a way to do so, and it is very hard to design a system that is resilient against that.

    Simple example: Everyone can give everyone else a score from 0 (worst) to 10 (best).

    Theory: While there are jerks who give bullshit scores, it will averag
    • At the moment, what I do is having no system.

      Actually, I do have something, but it's not a ranking system even close to what's discussed in the article. Anyways, here's how it works:

      Players can give "medals" to other players for positive things only. There are no negative marks. This removes the incentive for griefers because there's nothing bad they can do to anyone.

      The ability to give out medals depends on the number of medals you've received, so only people judged "good" by other people can give out meda
  • in wow ther is a freind list and an ignore list.. people you dont like go on the ignore list... its that simple.

Slashdot Top Deals

One man's constant is another man's variable. -- A.J. Perlis

Close