Each personality type can potentially irritate other people in their own particular way.
The potential ways in which an ISFP can irritate others include being too concerned with harmony in the group, not pushing the ISFP's own ideas and contribution enough, and eing stubborn over issues the group did not anticipate being a problem. By trying to avoid conflict you may fail to give (constructive) criticism when it is needed. You may also focus so much on interpersonal issues that objective considerations (e.g.: udgets, timescales, quality) take second place. You may also perhaps fail to take a longer term view and, taking people too much at face value, not recognise hidden agendas or motives.
You can overcome personality type differences by learning about your similarities and differences with other people. This leads to greater understanding, appreciation, and respect. Also, if you learn to adapt your style this will make you more influential and respected.
There is a comprehensive guide on how to do this in the book Influencing People Using Myers Briggs. This is available as an option when you complete the personality test.
Next: ISFP and Personal Growth