I think you will find there is very much to worry about, particularly if the rudder shaft where it extends down into the rudder is of stainless steel. I have mentioned in other posts elsewhere, that while stainless steel is great in most other areas, if it is allowed to remain in contact with stagnant sea water, which is what will happen here, it will corrode very rapidly. Best thing is to find where the water is leaking in, then drill holes in the rudder when the boat is next ashore, in order to allow any water to drain out. Give it time to thoroughly dry out, then fill the holes and glass over the leak and the area where you drilled holes. Best to speak to a qualified yacht surveyor, and chose one that has a reputation for being pernickety.
The following reference is in regard to stainless steel keel bolts, while we are talking about the rudder shaft and its attachment to the rudder. The relevance of course is in regard to what happens to stainless steel in a stagnant sea water environment.
http://www.sailmagazine.com/boatworks/how-secure-your-keelAbout halfway through the article is the following reference, but do read the rest of the article.
"Stainless steel: The most common keel-bolt material in America is stainless steel. This is dismaying to many, because stainless steel suffers rampant corrosion when immersed in stagnant seawater. Using it for keel bolts anticipates that the bolts will remain dry?which is optimism bordering on delusion. The good news is that if they do remain dry, they have proven to last for half a century or longer."
Your rudder shaft will not remain dry, while the water within the rudder itself will not be exchanged sufficiently frequent to re-oxygenate in order for the surface coating on the stainless parts to be regenerated. So don't be tempted to ignore the situation.