Each personality type can potentially irritate other people in their own particular way.
The potential ways in which an ISFJ can irritate others include avoiding conflict, not giving criticism when it is needed, and not articulating his/her understanding of the situation. You may be so focused on your own knowledge that you don't see the wood for the trees and you resist change (though not openly). You may also fail to look after your own needs, or not promote sufficiently your own ideas or contributions.
You can overcome personality type differences by learning about your similarities and differences with other people. This leads to greater understanding, appreciation, and respect. Also, if you learn to adapt your style this will make you more influential and respected.
There is a comprehensive guide on how to do this in the book Influencing People Using Myers Briggs. This is available as an option when you complete the personality test.
Next: ISFJ and Personal Growth