Well, not so much if it’s boring (that would be rude) but if you need to do something.  A lot of times people are under the impression that leaving the khutbah would be “impious” or “bad” but actually if you need to leave go right ahead.  The Prophet sallallaahu ‘alaihi wa sallam, when beginning the ‘Eid khutbah (not the regular one for Jumu’ah), would say, “Now we shall have a khutbah; so let whoever wishes to sit for the khutbah do so, and whoever wishes to leave leave.” (Muslim) It is good to listen to the khutbah because we all need reminders, but we shouldn’t censure or chastise people or look down on them if they want to leave.  The Prophet’s attitude, sallallaahu ‘alaihi wa sallam, was very different than that of many people’s nowadays where they pretty much beg you to stay and listen (well who can blame them - they took the trouble to sit and write up a reminder).  But it’s not the way of the Prophet sallallaahu ‘alaihi wa sallam.  This even went so far that some of the people started making the khutbah before the prayer! (They were however rebuked for deviating from the Sunnah - so Allah and His Messenger knew best.)

However, we have the other extreme where although people are not required to listen to the khutbah, they don’t follow the 2nd part of the Prophet’s statement “to leave”.  They basically just stand there chatting with their friends until no one can hear the khutbah!  So that is not proper either since the khutbah (if you choose to listen to it) is part of the prayer and you should not prevent anyone else from benefitting.  That is why subhanallah the Prophet sallallaahu ‘alaihiwa sallam in his beautiful wisdom and elegance just calmly told us the options- stay and listen but if you don’t want to then just leave so others can benefit. 

Your choice.