This paper discusses Bayesian inference in change-point models. Existing approaches involve placing a (possibly hierarchical) prior over a known number of change-points. We show how two popular priors have some potentially undesirable properties (e.g. allocating excessive prior weight to change-points near the end of the sample) and discuss how these properties relate to imposing a fixed number of changepoints in-sample. We develop a new hierarchical approach which allows some of of change-points to occur out-of sample. We show that this prior has desirable properties and handles the case where the number of change-points is unknown. Our hierarchical approach can be shown to nest a wide variety of change-point models, from timevarying parameter models to those with few (or no) breaks. Since our prior is hierarchical, data-based learning about the parameter which controls this variety occurs.