Would I be allowed to join him there?
Maybe. At article 3(2)(b), the directive includes among its beneficiaries "the partner with whom the Union citizen has a durable relationship, duly attested." The interpretation and application of this provision are left to each country, however, and I don't know what criteria Sweden applies to the recognition of such partners.
Registering our partnership is an option, but would that be enough?
Yes, if Sweden recognizes such partnerships as equivalent to marriage (which it seems to). If you do that, you will instead fall under Article 2(2)(b):
the partner with whom the Union citizen has contracted a registered partnership, on the basis of the legislation of a Member State, if the legislation of the host Member State treats registered partnerships as equivalent to marriage and in accordance with the conditions laid down in the relevant legislation of the host Member State;
You also write
I know that he would need to prove that he can sponsor me until I find a job, and that is not an issue.
You are wrong about that. If you fall under the directive, there are no financial requirements. The only requirement is that the principal be in one of the categories outlined in Article 7. Sweden is not very forthcoming about this in its website, but the application form (pdf) does reflect this.
We are more concerned about whether Sweden would recognize our relationship as a family tie valuable for migration.
Sweden seems to interpret "durable relationship" as implying cohabitation, which does not look good for you.
Also see the EU's page on unmarried couples, which notes that
If you move with your de facto partner to another EU country, that country must facilitate their entry and residence there - whether your partner is an EU national or not.
You must be able to prove you live together or that you are in a long-term relationship. However, most EU countries have not defined exactly how you can prove a long-term relationship or cohabitation.
