Only custody orders (as opposed to agreements entered into outside of court) can be modified through the court system.  If there is a custody order, these orders can be modified when one of the parents successfully shows that circumstances have substantially changed since the previous custody order and that those changes have affected the welfare of the child.  Examples of scenarios that potentially constitute a substantial change include changes in the child’s schedule (perhaps due to schooling), changes in either parents’ employment, changes in the location or nature of either parents’ residence, criminal activity by either party, loss of employment by either party, a parent’s failure to follow the existing court order, or either party’s ability or inability to spend time with and care for the child.

In the event that you never obtained a court order but want to modify your custody agreement, then both parties would need to voluntarily agree to change the original agreement, or one of the parties could file a custody action with the court to make an initial custody determination, at which point the court would make their decision using the best interest of the child standard (discussed above).