Local image histograms contain a great deal of information useful for applications in computer graphics, computer vision and computational photography. Making use of that information has been challenging because of the expense of computing histogram properties over large neighborhoods. Efficient algorithms exist for some specific computations like the bilateral filter, but not others. Here, we present an efficient and practical method for computing accurate derivatives and integrals of locally-weighted histograms over large neighborhoods. The method allows us to compute the location, height, width and integral of all local histogram modes at interactive rates. Among other things, it enables the first constant-time isotropic median filter, robust isotropic image morphology operators, an efficient “dominant mode” filter and a non-iterative alternative to the mean shift. In addition, we present a method to combat the over-sharpening that is typical of histogram-based edge-preserving smoothing. This post-processing step should make histogram-based filters not only fast and efficient, but also suitable for a variety of new applications.