It is not necessary for you to leave Germany. You can apply for a residence card at the local Ausländerbehörde for your spouse's place of residence.
Your question says that you are going back to India anyway, however, so you should apply for a family reunion visa with the appropriate German consulate in India. According to the application checklist, you'll need to submit evidence that your spouse has been working in Germany for the last three months. There are exceptions to that requirement under EU law, so if your spouse has not been working in Germany for the last three months, please post a comment.
Because you are applying as a family member of an EU citizen, the application and the visa will be free of charge, and it should be considered under an accelerated procedure. You will need to show your marriage certificate as part of the application.
Once you return to Germany, you will go to the Ausländerbehörde, as mentioned above, to apply for a residence card.
For example, here is the relevant page of the Ausländerbehörde in Hamburg:
The freedom of movement rules are set forth in directive 2004/38/EC; there are also a summary of the legislation and a public information page describing the situation.