I often need to describe to people the difference between reverence and irreverence by making a comparison with various events in our lives. I used to use an illustration where I would ask people "you wouldn't want to go to a formal event, or meet a king, without being dressed nicely, right?" Years ago, everyone would say "of course not". Over time, that diminished more and more, and an increasing number of people began to say "I don't care what others think, this is me, so you can take it or leave it". At first it was just depressing to see that people really had so little respect for themselves or others around them. Lately though, this has become the majority who respond this way and it more than depressing.
A deeply selfish sense of disrespect has spread throughout our society like a disease. You can see it in more ways that I want to recount here. What drives this attitude is not always the same for each person, but it is always tied to a prideful heart and mind that says "I matter first, and everything else is insignificant" (rarely do I actually hear people say this, but I have heard it a few times -- and sometimes from those who claim to be Christian!).
How casual is too casual? The most common answer to that question would be different in our day than it was just 70 years ago. In 1950 men went to a party in a suit and tie. Go back a few hundred years and it would different again. I am not implying that standards are completely relative, but rather that our definition of "casual" will change over time. It is self evident that we live, today, in a radically casual age. Just look at the most popular styles of clothing in stores; yes, there are still formal things being sold, but they are often considered "stuffy" (and some clothing stores do not carry anything considered "dressy").
Going back to the point of my old illustration that I mentioned above: there are a few things that are just basic courtesy when I comes to formality. Firstly, how we dress shows whether we respect ourselves; secondly, how we dress shows the respect that we have for others; and thirdly, how we dress shows our respect for God. Yes, that third one may surprise a few people, but it is true. After all, God is the One Who created us, and He is the One Who told us to cover our nakedness. This means that we should take into consideration (at the very least) that He cares how we clothe ourselves.
God made clothing for Adam and Eve; He gave numerous rules for the priests of the Old Covenant for how they were to dress (which is why we have rules for Catholic priest's clothing), and there are quite a few places where rules for clothing (usually for women) show up in the New Covenant (cf. 1 Cor 11:4ff, 1 Tim 2:9, 1 Pet 3:3). God cares how we dress ourselves; we cannot consider it a small matter.
Drawing from this, we can see that of all the places where we go, we should be concerned with how we dress when in God's presence. In spite of the foolish answers given by many today, yes, we should care how we dress in the presence of a high dignitary, and therefore, we should care how we dress in the presence of our Almighty Lord. The Mass is certainly not a fashion show, but it is a place where we should show some basic self-respect. This implies that things like shorts, flip flops, and shirts with pictures and advertisements are never proper for the worship of God (and though it is not a piece of clothing -- chewing gum in Church is comparable to the most disrespectful clothing imaginable [and it counts as breaking the fast before communion!]). These things are what we call "casual".
It is interesting to note that the word "casual" and "casualty" both come from the same Latin root that means "by chance" or "out of control". A person who dresses casually at a formal event is lacking in self-control and cannot be relied upon. A person who dies (a "casualty") came to their end by an event outside themselves and was not in control of their situation. In other words, a lack of self-control in one's dress reveals a deeper lack of self-control in their hearts. One who wishes to express self-control and trustworthiness will dress appropriately to the situation (for the sake of one's own personal convictions as well as the expression of concern for others).
When a society starts to go off the rails by rejecting the worship of Almighty God, then it will also reject reverence of any kind. A lack of reverence in clothing will always exemplify one's lack of deeper reverence toward God Himself. It will also show itself in lack of respect in how he speaks to others (i.e. disrespectfully), whether he uses proper titles for others (e.g. sir, ma'am, officer, Mr. President, etc.). It will show itself in table manners (presuming they were ever taught in childhood).
It is not surprising that I have never seen someone who comes to Mass all decked out in "casualness" to show genuine reverence. They tend to be resistant to genuflecting, or speaking in quiet tones while in the Church. They are often the same ones who "lounge" in the pew during Mass and show signs of boredom about five minutes into the Mass. At the same time, those who choose to dress more respectfully are also the ones who are more attentive during the Mass and show a clear desire to respect the things of God in every action. The outward influences the inward; always!
What is your goal? Reverence? Respect? Or is it being comfortable and casual? You do not need to spend a ton of money to look nice for the Lord, but you do need to have a reverent heart in order to be concerned about how you approach God. It is not to show to others that you "look nice" so that you can get a few compliments, but rather to show to God that you respect Him and want to present yourself, consistently, with respect and reverence to honor Him. Let us leave casual dress for its proper place and stop showing disrespect for God and the things of God. Let us love the Lord with all our heart, soul, mind and body.