This is a splendid question, one which I had not thought of before. There are countless factors that go into making the "inscrutable character," but hopefully I will be able to touch adequately upon a few of them. Disclaimer: in writing this post, my subconscious yanked on my sleeve and reminded me that I like to write big, red-blooded people, not teenage girls in high school, and that severely dictated the tack I chose to take while writing this piece. I have worn down the welcome-mats of the minds of Hercules and Adonis; the brain of the average teenage girl remains shrouded in mystery, even for me.
First of all, as a way of limbering up to this new thought process, please remember that people are contradictory and confusing, and while you certainly don't want to make your characters bipolar (unless you intentionally want to write a bipolar character), it makes them realistic to allow them those contradictory beliefs, thoughts, emotions, and actions with which real people are plagued every day. Pay attention to that as the writer, but do try to give them as unstudied an air as possible.
the great man
In a conversation with my father about keeping one's mouth shut, he remarked that he considered that great men have often been those who kept their own council. I had to agree: the people I have looked up to in literature have been those whose gravity and silence have unintentionally lent them what some people might call an aloof air. They were not blabs. They kept their word, and they did not speak their words often. They had integrity because of that, which is something that is often remarked upon in the chapters of Proverbs - and, indeed, is a virtue to be found in all cultures. This is a trait that is independent of personality. Some personalities will naturally tend against the keeping of one's council, but it can be done. Protagonist and villain alike may express this virtue. I know that mine have. My protagonist of Adamantine is a great one for keeping his thoughts to himself. My villain of Plenilune, also, has the habit of holding his cards close to his chest.
the wild card
Some characters will keep their inside selves secret by presenting a different face to the world. This is called duplicity. Sometimes it is employed to keep the character safe, sometimes it is employed to save others; I have used both methods. Sometimes a character is unaware of this Janus aspect: this is called irony. And this is where the real problem begins: how do you portray these two sides in these situations?
The back pocket character. This is a ploy I use most often. When I have a character such as the "great man," who plays his cards very close to his chest, I need to have an intimate character to come alongside him and interpret for the reader. They are generally good friends, or the one is a great admirer of the other and able to be often in that character's presence, enough to get to know that character to a degree unknown to the rest of the people in the book. To the others, the "great man" may be silent, brooding, distant, abstruse, bewildering, eccentric. He is looked upon with a kind of mingled awe and misapprehension. To the friend he is in many ways understood, and in those ways that he is not understood he is forgiven. This method is used of the famous Artos in Watch Fires to the North. The book is written from the character Bedwyr's point of view, and so the reader is allowed an outside view of the character Artos. It would overbear the book too much to get into the great man's mind: we are allowed to see him through his friend's eyes, and so, in a way, we are able to come to an understanding.
The first person. In the first person, you get a front-row seat of the character's thought processes. While he reasons why he is being duplicitous, the reader will naturally be abreast of the situation. This can be seen in a novel like C.S. Lewis' Till We Have Faces - an excellent book, full of inner turmoil and duplicity masterfully delivered to the reader."I have perhaps drawn his portrait clumsily, but he was not an easy man to know."
Making the reader the back pocket character. I use this ploy in The Shadow Things and Between Earth and Sky. While the main characters of both stories have good friends, in their way, there is no one quite equal to share the characters' thoughts and so the reflections are divulged through the process of third-person past-tense prose to the reader. There is a great deal of introspection (though not, I trust, in the dull and drawn-out way that is like being tortured); when the character does do something spontaneous, the reader has come to know him so well that no explanation is needful. The reader has become the character's friend.
Keep in mind the reason for the duplicity. Unless the character is naturally quiet, there is usually a good reason for the character to keep his mouth shut. A character of mine was badly betrayed: he has created the habit of keeping himself to himself to avoid being hurt, and laughing to cover the pain. He has a naturally spirited disposition which demands a kind of outlet, but he has also a great reason for anger and resentment - which makes the "wild card" aspect of what he might say or do all the more interesting for on-lookers and reader alike!
Keep in mind the reveal. If you do plan on pulling the sheet off your character's duplicitous nature, foreshadowing is a great ploy. Everybody loves it. Drop a few hints! Make another character suspicious! You choose!
We are all confusing, contradictory, duplicitous, and ironic at times. Our characters will be also. This is a good thing! But the base of all these reactions, I have found, to make a situation comprehensible to the reader, is friendship. It is impossible to understand anyone apart from some degree of friendship. It may be another character, it may be the reader, but someone must befriend the inscrutable character to make any sense out of the inscrutable mind.