Young supernova remnants (SNRs) are often assumed to be the source of cosmic rays up to energies approaching the slight steepening in the cosmic-ray spectrum at around 1000 TeV, known as the ``knee.'' We show that the observed X-ray emission of 14 radio-bright shell remnants, including all five historical shells, can be used to put limits on E_max, the energy at which the electron energy distribution must steepen from its slope at radio-emitting energies. Most of the remnants show thermal spectra, so any synchrotron component must fall below the observed X-ray fluxes. We obtain upper limits on E_max by considering the most rapid physically plausible cutoff in the relativistic electron distribution, an exponential, which is as sharp or sharper than found in any more elaborate models. This maximally curved model then gives us the highest possible E_max consistent with not exceeding observed X-rays. Our results are thus independent of particular models for the electron spectrum in SNRs. Assuming homogeneous emitting volumes with a constant magnetic field strength of 10 μG, no object could reach 1000 TeV, and only one, Kes 73, has an upper limit on E_max above 100 TeV. All the other remnants have limits at or below 80 TeV. E_max is probably set by the finite remnant lifetime rather than by synchrotron losses for remnants younger than a few thousand years, so that an observed electron steepening should be accompanied by steepening at the same energy for protons. More complicated, inhomogeneous models could allow higher values of E_max in parts of the remnant, but the emission-weighted average value, that characteristic of typical electrons, should obey these limits. The young remnants are not expected to improve much over their remaining lives at producing the highest energy Galactic cosmic rays; if they cannot, this picture of cosmic-ray origin may need major alteration.