Can a robot learn to grasp new objects without retraining? This research proposes a deep learning architecture with augmented memory to handle open-ended object recognition and grasping simultaneously. The robot is able to use multi-views of an object as input and jointly estimates pixel-wise grasp configuration as well as a deep scale- and rotation-invariant representation as output. This approach uses meta-active learning for open-ended object recognition, resolving the issue of catastrophic forgetting when encountering new object categories. The obtained representation is then used for open-ended object recognition through a meta-active learning technique. This enables a robot to continuously acquire knowledge about new object categories. The approach allows the robot to learn new object categories rapidly using very few examples on-site, demonstrating effective grasping of never-seen-before objects. The system shows high object recognition accuracy and grasp success rates in cluttered environments, empowering robots to adapt and acquire new knowledge effectively.