Step 5: child benefit/tax credits (within first 3 months)
I was not able to do this till my immigration status application for Indefinite Leave to Remain was accepted, so we only just became eligible for this.
Child benefit: the form arrives with your paperwork from the hospital.
Fill it out and send in your baby's birth certificate and (if your immigration status has changed in the last 3 months) your biometric card
Tax credits: Call their helpline to ask for a form. They will ask you questions. If your husband works for the government as a civil or public servant you will be reconnected to a hundred different people and your husband will have to be interviewed in person. Then they send you the form.
Fill it out and send your baby's UK passport (if they do not have two British parents).
By post: I was not comfortable sending important documents in normal post so I got special delivery for all these. I sent self-addressed special delivery envelopes enclosed as well.
Then after about a month of waiting, all our documents were returned and money sent to our bank account. Thank you, UK government!