This paper presents a new technique for achieving spectral accuracy and fast computational performance in a class of black hole perturbation and gravitational self-force calculations involving extreme mass ratios and generic orbits. Called spectral source integration (SSI), this method should see widespread future use in problems that entail (i) point-particle description of the small compact object, (ii) frequency domain decomposition, and (iii) use of the background eccentric geodesic motion. Frequency domain approaches are widely used in both perturbation theory flux-balance calculations and in local gravitational self-force calculations. Recent self-force calculations in Lorenz gauge, using the frequency domain and method of extended homogeneous solutions, have been able to accurately reach eccentricities as high as e≃0.7. We show here SSI successfully applied to Lorenz gauge. In a double precision Lorenz gauge code, SSI enhances the accuracy of results and makes a factor of three improvement in the overall speed. The primary initial application of SSI (for us its raison d'etre) is in an arbitrary precision Mathematica code that computes perturbations of eccentric orbits in the Regge-Wheeler gauge to extraordinarily high accuracy (e.g., 200 decimal places). These high accuracy eccentric orbit calculations would not be possible without the exponential convergence of SSI. We believe the method will extend to work for inspirals on Kerr, and will be the subject of a later publication. SSI borrows concepts from discrete-time signal processing and is used to calculate the mode normalization coefficients in perturbation theory via sums over modest numbers of points around an orbit. A variant of the idea is used to obtain spectral accuracy in solution of the geodesic orbital motion.