In this paper, we discuss the singular characteristics of the Internet of Things (IoT) in regard to security issues and present a security architecture with special focus on Edge networks and Smart Things. The architecture is based on the assumption that data integrity, authentication, and service/device access control are the most important aspects that have to be guaranteed for safe operation of highly decentralized and heterogeneous IoT systems. The architecture is made concrete as an extension of the ContextNet middleware for IoT. This middleware has a cloud and a mobile component and advocates the use of smartphones - or devices that execute smartphone platforms - as the universal gateways between cloud-hosted IoT services and the Smart Things. In addition to general architecture, the other major contribution is a protocol for establishing a secure connection between the Smart Things and these mobile gateways.