Congress rejected the invitation to inaugurate Ram temple