Based on comments on your question, it appears that you do not need to renew your US visa, and your question is based on your incorrect belief that a US visa is needed for you to stay in the US, which is completely false. A US visa is solely for entry to the US. That is the reason that a US visa can only be gotten outside the US (because if you are already in the US, you do not need to "enter"). You will only need to renew your US visa if you plan to travel outside the US and return to the US after your current visa expires.
Your ability to stay in the US is determined by your status (in your case, H-1b status), the duration of which is determined by your I-94. H-1b status usually extends to the end of validity of your H-1b petition. If the time given on your I-94 is expiring soon (which you have no indicated is the case), your company should apply to extend your H-1b petition and status. Once the application to extend your status is filed, you can continue to stay and work in the US while the application is being processed; and when it is approved, you will receive a new I-94 which determines the new extent of your status.
The next time you leave the US to go back to New Zealand or go to somewhere else (however long in the future that is), and need to come back to the US to continue to work, you will need to get an H-1b visa at a US consulate abroad, if they accept visa applications from non-residents. If you do not need to leave the US, you do not need to get a US visa, ever.
Even if you leave the US to try to get an H-1b visa abroad, you will need a valid H-1b petition to get an H-1b visa, and when you come back to the US, your I-94 will be limited to the validity of your H-1b petition, so if your petition is expiring, then you will need your company to apply to extend your H-1b petition anyway; otherwise, getting an H-1b visa and/or re-entering the US will be completely useless, as it will not extend the amount of time you can stay in the US beyond what you can currently. And if the petition is already expired or in the process of being extended, you will not be able to get an H-1b visa abroad until the new petition gets approved, which could take a long time.
The only cases in which one might want to get a US visa in a nearby country, is 1) if you have plans to travel abroad, but the time in that country is too brief to apply for a US visa at the US consulate there (e.g. an academic conference), or the consulate there does not accept visa applications from non-resident applicants; or 2) you need to apply for a visa to a third country, which requires you to have a valid US visa before they will issue you their visa (not many countries have such a requirement). In these cases, people commonly go to US consulates in Canada or Mexico, which are generally the most convenient to reach from the US. Caribbean countries might work similarly.