Despite a strong fourth quarter, many U.S. bankers foresee a worsening economy and higher funding costs.