The damping properties of materials, joints, and assembled structures can be modeled efficiently by the use of fractional derivatives in their constitutive equations. The respective models describe the damping behavior accurately over broad ranges of time or frequency where only few material parameters are needed. They assure causality and pure dissipative behavior. The concept of fractional derivatives can be implemented into discretization methods such as the finite element method, the boundary element method, or the finite difference method. Due to the non-local character of fractional derivatives the whole deformation history of the structure under consideration has to be considered in time-domain computations. This leads to increasing storage requirements and high computational costs. A new concept for an effective numerical evaluation makes use of the equivalence between the Riemann-Liouville definition of fractional derivatives and the solution of a partial differential equation (PDE). The solution of the PDE is found by applying the method of weighted residuals where the domain is split into finite elements using appropriate shape functions. This approach leads to accurate results for the calculation of fractional derivatives where the numerical effort is significantly reduced compared with alternative approaches. Finally, this method is used in conjunction with a spatial finite element discretization and a simple rod structure is calculated. The results are compared to those obtained from alternative formulations by means of accuracy, storage requirements, and computational costs.