Per Investopedia, loan servicing includes sending monthly payment statements, collecting monthly payments, maintaining records of payments and balances, collecting and paying taxes and insurance (and managing escrow funds), remitting funds to the holders/owners of the mortgage, and following up with any delinquencies.
One of the things that sometimes confuses borrowers is the fact that servicers are often separate entities from the mortgage holders or the investors that buy mortgage loans.
When a mortgage bank like JVM/American Pacific Mortgage (APM) funds a mortgage it has to sell the loan at a premium shortly after funding in order to make money.
When mortgage banks sell their loans, however, they can hold on to the servicing themselves, sell the loans with the servicing rights, or sell the loans separately from the servicing rights.
A mortgage bank might hold onto servicing rights if it wants to collect the fee income itself over time, but more often than not mortgage banks sell servicing rights when they sell their loans.
While JVM/APM does retain some servicing, we either opt to sell or are forced to sell servicing rights in many cases, depending on loan type.
The investors that buy our jumbo loans, for example, typically demand servicing rights along with the loan (so we are not allowed to service them even if we want to).
Borrowers also sometimes request that we avoid specific servicers because they had experiences with them in the past.
Unfortunately though, while mortgage banks can sometimes avoid servicers with particularly poor reputations, they more often than not have no control over who ends up handling the servicing.
This is particularly the case when servicing rights get sold or transferred more than once after a loan is sold.