What's the deal with speaking in tongues?

So you may be familiar with this Christian practice. It may also seem extremely strange for a group of people to be making sounds something like "Walla walla durka durka durka" and thinking that it's ok. You wouldn't be alone in thinking that. Even Paul the Apostle said:

If, therefore, the whole church comes together and all speak in tongues, and outsiders or unbelievers enter, will they not say that you are out of your minds? -- 1 Corinthians 14:23

But even for all the admitted weirdness, it is something that has a Biblical basis. '1 Corinthians 12-14' covers much of what tongues is about, but it also crops up elsewhere too.
The first time is in 'Acts 2' when the apostles gather for Pentecost. In verse 4 the Holy Spirit comes to them and allows them to speak in tongues. The following section details how there were Jews around from all over the world:

Parthians and Medes and Elamites and residents of Mesopotamia, Judea and Cappadocia, Pontus and Asia, Phrygia and Pamphylia, Egypt and the parts of Libya belonging to Cyrene, and visitors from Rome, both Jews and proselytes, Cretans and Arabians—we hear them telling in our own tongues the mighty works of God. -- Acts 2:9-11

The people from these various nations could understand what the apostles were saying in their own languages.

And at this sound the multitude came together, and they were bewildered, because each one was hearing them speak in his own language. -- Acts 2:6

So, we see very clearly that in this instance, the gift of tongues was given so that the people around the apostles could clearly understand the Gospel message.
Then it goes on:

And all were amazed and perplexed, saying to one another, “What does this mean?” But others mocking said, “They are filled with new wine.” -- Acts 2:12-13

So it seems that some people at least could not understand the tongues. Their best guess was that they were drunk. On hearing this, Peter sets things straight by speaking in plain language:

Men of Judea and all who dwell in Jerusalem, let this be known to you, and give ear to my words. For these people are not drunk, as you suppose, since it is only the third hour of the day. But this is what was uttered through the prophet Joel -- Acts 2:14-16

Then he goes on to deliver the same message that was being given in tongues, but in his native language so that everyone could understand clearly. After being given the message, the people "devoted themselves to the apostles' teaching and the fellowship, to the breaking of bread and the prayers." (Acts 2:42)

This event sums up the proper use of the gift of tongues extremely well. At first the gift is received as a miraculous sign for the apostles. They feel the Holy Spirit working with them. Then the people around them who can see the miracle happening and benefit from the message in a language they can understand are also impressed and awed by what is happening. Not only do they hear God's good news, but they witness his power in action. So this is encouraging for both the apostles and the people listening.
But then we find some people who aren't benefiting from the tongues. They are being left out and start to think that the apostles are crazy. At that point, Peter stops speaking in tongues and goes back to his normal language. He does this so that he can share the message with them too. It brings them into the group, where they were previously being left out.
The purpose of the tongues was to build up the relationship between the apostles and some of the people. But as soon as that purpose had been served, the tongues were stopped so that everyone could be included.

This picture is a far cry from the people who think tongues are something you do loudly in church while everyone around you is left to think you are having a personal moment. Tongues aren't just gobbledegook meant to be understood only by the Holy Spirit. Their dual purpose is specifically to bring messages to people and to build up people's faith.

So let's look into '1 Corinthians 12-14'. In these chapters Paul describes the proper place for spiritual gifts. Tongues is a prime topic. He essentially covers what we have already seen practised in Acts by Peter, but there are more details and clarifications.

Now concerning spiritual gifts, brothers, I do not want you to be uninformed. -- 1 Corinthians 12:1

In '12:4-11' Paul tells us that the Holy Spirit offers a number of different gifts. Not everyone gets them all, but whatever gifts are given are meant for service: wisdom, knowledge, faith, healing, working of miracles, prophecy, the ability to distinguish between spirits, tongues, the interpretation of tongues.
In '12:12-26' Paul describes the way that different gifts are meant to complement the gifts of other people. He uses the human body as an analogy. We have eyes to see, and hands to touch. That means hands are helpful to eyes. But if you think sight is the only important body function, then you will think hands are useless because they can't see. This analogy tells us that different gifts are all important in their own ways, but we should not think of one particular gift as being the only important thing. However he does recognise that some body parts are naturally stronger. Despite some being stronger, it is also true that the seemingly weaker parts are indispensable. Which leads into verse 28, where he lists a ranking of gifts from most important to least. So the point is that all of these gifts are vital, but even within that some are more important than others.

And God has appointed in the church first apostles, second prophets, third teachers, then miracles, then gifts of healing, helping, administrating, and various kinds of tongues. -- 1 Corinthians 12:28

You'll notice that tongues come in last on the list. But Paul's warning is that just because something is lower in the ranks, it doesn't mean it should be thought of as less vital. However, it also means that a gift low on the list should not try to hold it proudly over other higher gifts. Just like at Pentecost, all the gifts are meant to build up the community.

If one member suffers, all suffer together; if one member is honoured, all rejoice together. -- 1 Corinthians 12:26

He goes on in the next chapter.

If I speak in the tongues of men and of angels, but have not love, I am a noisy gong or a clanging cymbal. -- 1 Corinthians 13:1

He clearly is well aware of what tongues seem like to outsiders. We see here 'the tongues of men', which clearly means languages that people speak and understand. Remember in '1 Corinthians 12:10' Paul referred to 'different kinds of tongues'. We also see here 'tongues of angels'. So when tongues are spoken and nobody understands, it could be that the speaker is using an angelic language. 
The next passages describe love:

Love is patient and kind; love does not envy or boast; it is not arrogant  or rude. It does not insist on its own way; it is not irritable or resentful; it does not rejoice at wrongdoing, but rejoices with the truth. Love bears all things, believes all things, hopes all things, endures all things. -- 1 Corinthians 13:4-6

This links in to the previous section where he warned us not to use our gifts to show off or to think we are better than others with lesser or no gifts at all. The rest of Chapter 13 reminds us that all of these gifts are temporary things. They will not be needed forever and will eventually pass away. But the one thing that stays and is never lost is love. That should be the focus, and our gifts are meant to be used in loving ways.

In Chapter 14, Paul starts explaining this different type of tongue. 

For one who speaks in a tongue speaks not to men but to God; for no one understands him, but he utters mysteries in the Spirit. -- 1 Corinthians 14:2

This must be the 'tongues of angels'. We previously saw at Pentecost that people understand the tongues of men, but here Paul tells us that men won't understand these tongues (unless perhaps there is someone who has the gift of interpretation).
So if someone speaks in the tongues of angels, they are speaking directly to God through the Spirit. He then compares this type of speaking in tongues to prophecy.

On the other hand, the one who prophesies speaks to people for their upbuilding and encouragement and consolation. The one who speaks in a tongue builds up himself, but the one who prophesies builds up the church. Now I want you all to speak in tongues, but even more to prophesy. The one who prophesies is greater than the one who speaks in tongues, unless someone interprets, so that the church may be built up. -- 1 Corinthians 14:3-5

He makes a very clear statement that in church, prophecy is much better than tongues of angels. Tongues of angels are something specifically for the person speaking them, while prophecy is for the congregation. Except again, unless there is an interpreter available. In that case, tongues of angels can be something to share with the church.
In verses 6-12, Paul again compares tongues that nobody understands to senseless noise. He realises that the words spoken in tongues have meaning, but if nobody knows the meaning, then it's pointless to make the noise.
Paul goes on to tell us that he has the gift of tongues, but not the gift of interpretation. When he prays in tongues, he knows that the Spirit is praying for him, but he has no idea what is being communicated. If he was to do that out loud, he would be sharing that confusion with everyone around him instead of just keeping it to himself. So he prefers to pray in his own language, because he understands it, and he can share the prayers with his disciples so that they can benefit from them too.

in church I would rather speak five words with my mind in order to instruct others, than ten thousand words in a tongue. -- 1 Corinthians 14:19

Paul recognises the value of tongues, but tells us that their purpose is to strengthen faith in the church. If it is not being used to do that, you should stop, and keep it personal.

Unless you speak intelligible words with your tongue, how will anyone know what you are saying?... Otherwise when you are praising God in the Spirit, how can someone else, who is now put in the position of an inquirer, say “Amen” to your thanksgiving, since they do not know what you are saying? -- 1 Corinthians 14:9,16

In verses 20-25, Paul clarifies a misunderstanding in the Corinthian church. He quotes the Law and how they have interpreted it:

In the Law it is written, “By people of strange tongues and by the lips of foreigners will I speak to this people, and even then they will not listen to me, says the Lord.” Thus tongues are a sign not for believers but for unbelievers, while prophecy is a sign not for unbelievers but for believers. -- 1 Corinthians 14:21-22

He then tells them what they should really believe.

If, therefore, the whole church comes together and all speak in tongues, and outsiders or unbelievers enter, will they not say that you are out of your minds? But if all prophesy, and an unbeliever or outsider enters, he is convicted by all, he is called to account by all, the secrets of his heart are disclosed, and so, falling on his face, he will worship God and declare that God is really among you.
-- 1 Corinthians 14:23-25

What Paul explains is that although you might think that to an outsider tongues are an amazing miracle that prove God's love, what is more likely to happen is that they will think you are crazy. You're more likely to impress and convince a non-believer with prophecy and teaching in plain language.
Paul finishes the Chapter with a summary of the ways to correctly use gifts. Regarding tongues these are his points:

• Let all things be done for building up. (14:26)
• If any speak in a tongue, let there be only two or at most three, and each in turn, and let someone interpret. (14:27)
• But if there is no one to interpret, let each of them keep silent in church and speak to himself and to God. (14:28)
• So, my brothers, earnestly desire to prophesy, and do not forbid speaking in tongues. But all things should be done decently and in order.  (14:30)

So a warning is given for those people who speak in tongues loudly in church. Basically you should not do it, unless there is someone who can interpret and it can be used to build up the congregation. Save it for your private prayers.

Another example of speaking in tongues in this way is seen in 'Acts 10:46' where Peter hears Gentiles who had just received the Holy Spirit speaking in tongues and praising God. Seeing as he understands what they were doing, it must be that he could somehow interpret the tongues.
There is also 'Acts 19:6' where twelve men in Ephesus were baptised and the Holy Spirit gave them the gifts of tongues and prophecy.

So clearly, the gift of tongues is a great thing, and a sign from God that he is with you. But not everyone gets this gift, and that is no big deal. There is no expectation for Christians to be able to ever speak in tongues, either those of men or angels.

However there is one last concern that Paul only mentions in passing. This is right at the beginning of '1 Corinthians 12'.

You know that when you were pagans you were led astray to mute idols, however you were led. Therefore I want you to understand that no one speaking in the Spirit of God ever says “Jesus is accursed!” and no one can say “Jesus is Lord” except in the Holy Spirit. -- 1 Corinthians 12:2-3

There were and are many pagan religions that have a practice that seems similar to Christian tongues. This is called Glossolalia. It's basically noises that might sound a bit like a foreign language, but are actually nonsense and completely made up, usually in a state of euphoria or trance or some kind of emotional high that might feel like it comes from God. Anyone can learn to do it, and a clue that Christians are probably practicing Glossolalia rather than actual Spiritual Christian Tongues is that they have been coached by someone else. Remember, that tongues are a gift from the Holy Spirit, not something that you can learn to do. Glossolalia has more in common with Tourette's Syndrome than a spiritual gift.

Paul in the above verses refers to these. A pagan could in their Glossolalia say bad things about Jesus, but someone praying in the Spirit could never do that. So this is another reason to be aware of if the tongues you are speaking are not being understood by anyone around you. It could be that you are unknowingly doing a pagan ritual, and not actually praising God as you think you are. Generally this mistake is probably harmless, but there is also the concern that in opening yourself up to these pagan practices, you are giving Satan an invitation into your life.

To Sum Up

So let's take what we've seen and put it into easily digestible bitesize chunks that you can wrap your tongue around.

• Tongues are a gift from the Holy Spirit. (Acts 2:4, 1 Corinthians 12:4-11)
• There are two different types of gifts of tongues: tongues of men, tongues of angels.
• At Pentecost, when the Apostles were given the gift of tongues, they spoke in known languages to people who spoke those languages. It was not gobbledegook. These were tongues of men. (Acts 2:6-11)
• When people did not understand the tongues, the Apostles would no longer use them, and speak in plain language so that people around would understand. (Acts 2:12-42)
• Tongues are one of many potential gifts from the Holy Spirit. They are regarded as the least important among the others, but are no less vital. (1 Corinthians 12:12-28)
• Only some people receive this gift, but that does not make them more important than anyone else.
• Our gifts are meant to be used to support each other and make the church stronger as a whole. (1 Corinthians 12:24-27) 
• All gifts are meant to be used only for loving reasons. There is to be no boasting, arrogance, or any ideas that your gift makes you more important than other people. (1 Corinthians 13)
• If nobody in a church understands tongues that are being spoken, it is possible that the speaker is using tongues of angels. These are words spoken directly to God. (1 Corinthians 14:2)
• If nobody around you understands your tongues, you should either stop and speak in normal language, or do it silently as a personal prayer. (1 Corinthians 14:16-17, 27-28)
• Be aware that if you are speaking in tongues and nobody understand you, you may not actually be speaking in tongues at all.