It sounds like your closer has a fast swing speed and a more normal latching speed. It also sounds like you probably know how to adjust the swing speed, but it will not adjust. If these two assertions are true, perhaps the swing speed valve seal is failing or the valve itself is worn out. This typically occurs in closers that have been in service for several years, or in closers that are defective. Another possibility is that the closer has lost some of its hydraulic fluid, but since the fluid usually drips down the face of the door when it escapes, this cause is easier to detect.
If the seals or valves are failing, or if the door closer is leaking, it is probably time to replace or rebuild the door closer.