Given data:
The mass of ball 1 is m.
The mass of ball 2 is 1.26m.
The initial speed of ball 1 is u=5.4 m/s.
The final speed of the ball 1 U=2.6 m/s.
The angle at which the ball 1 moves from x-axis is θ=36.9.
Applying the conservation of momentum in x-direction,
[tex]\begin{gathered} mu=mU\cos \theta+(1.26m)V\cos \alpha \\ u=U\cos \theta+(1.26)V\cos \alpha \\ 5.4=2.6\cos 36.9+(1.26)V\cos \alpha \\ V\cos \alpha=2.63\ldots\ldots\text{.}(1) \end{gathered}[/tex]Here, V is the final speed of ball 2, and α is the angle of ball 2 with x-axis after the collision.
Applying the conservation of momentum in y-direction,
[tex]\begin{gathered} 0=mU\sin \theta+(1.26m)V\sin \alpha \\ 0=U\sin \theta+(1.26)V\sin \alpha \\ 0=2.6\sin 36.9+(1.26)V\sin \alpha \\ V\sin \alpha=-1.56\ldots\ldots\text{.}(2) \end{gathered}[/tex]Dividing equation (2) and (1),
[tex]\begin{gathered} \frac{V\sin \alpha}{V\cos \alpha}=\frac{-1.56}{2.63} \\ \tan \alpha=0.593 \\ \alpha=30.6\degree \end{gathered}[/tex]Subsitute the value of α in equation (1),
[tex]\begin{gathered} V\cos \alpha=2.63 \\ V\cos 30.6\degree=2.63 \\ V=3.05\text{ m/s} \end{gathered}[/tex]Thus, the final speed of the ball 2 (orange ball) is 3.05 m/s, and the direction of the orange ball is 30.6⁰.