Community

Topic: Xamarin component

Issue(s) getting Pikkart running in Xamarin Forms app (SOLVED)

Dominic Jones
Thursday, February 18, 2021

Hi,

I'm trying to get Pikkart running (just the simple image tracking at the moment) in a Xamarin Forms app - I specifically want to get it running on a particular page in the Views section of the cross-platform app.

I'm currently targeting Android only.

Have managed to tweak the Xamarin guide to get it to compile, but it now stops all UI elements from Forms appearing and does not seem to be picking up markers - no alerts show.

So, I'd like some help if possible to get it running correctly and have it only active when the ARSCreen() page is pushed onto the top of the navigation stack.

A link to the GitHub repo for the test app is here: https://github.com/TheDapperLab/ShaPlay3

Many thanks in advance,
Dom

Dominic Jones
Monday, March 1, 2021

Update:

I've reconfigured the solution to use Dependency Injection, which has solved the UI issue and getting AR running on a specific page, but it now just crashes as soon as I launch the AR component...

Help urgently required and gratefully received!

user profile image
Support Team  (expert)
Tuesday, March 2, 2021

Hello,
can you tell me what error you get?

The RecognitionFragment fragment only performs image recognition but does not render anything, to do this you need an ARView that renders the camera and the digital elements.

From the github repository I see that the ARinterface activity adds the fragment RecognitionFragment but there isn't any ARView to render the camera. At this link there is a guide with the classes to add to create an ARView.

https://developer.pikkart.com/servizi/Menu/dinamica.aspx?idSezione=617&idArea=619&idCat=628&ID=1975&TipoElemento=page

However let me know with what error the app crashes.

Thank you.

Dominic Jones
Tuesday, March 2, 2021

Hi,
Thanks for getting back to me - very much appreciated!
The issue I'm getting is an exception thrown when I try to startRecognition on the cameraFragment. The exception is a null reference:
System.NullReferenceException: 'Object reference not set to an instance of an object.'
So... I'm presuming that the fragment is not registering correctly from the ARScreen XAML and/or I'm calling FragmentManager on the wrong object ('this', but inside the Native interface class).
Any ideas?
I'm not yet trying to render an AR VIew, just getting the camera recognition running as per the Android Xamarin Tutoral - AR View is next up!
Any and all help gratefully received - I don't know much about Native app dev, so I'm sure I'm probably doing/not doing something simple, somewhere...
Many thanks,
Dom

user profile image
Support Team  (expert)
Tuesday, March 2, 2021

Hi,
I looked at the repository and the problem here is that the activity is never launched and is not created before starting the recognition.

The change I would make is to instead of calling a method of the activity directly through a dependency service, use the dependency service to do the LaunchActivity causing the execution of the lifecycle of an Activity.

In the ShaPlay3 forms project i created an interface called ILaunchActivity which has the method LaunchActivityInAndroid:


public interface ILaunchActivity
{
void LaunchActivityInAndroid();
}


in the SplashScreen.xaml.cs class instead of replacing the MainPage with ARScreen I managed to call the LaunchActivityInAndroid method through a dependency service


private void StartButton_OnClicked(object sender, EventArgs e)
{
//Application.Current.MainPage = new NavigationPage(new ARScreen());

ARengine = DependencyService.Get();
ARengine.LaunchActivityInAndroid();
}


in the ShaPlay3.Android project I added a LaunchActivity class as a depencency service which extends the ILaunchActivity interface and which implements the LaunchActivityInAndroid method
in this method I launch the ARInterface activity.


[assembly: Xamarin.Forms.Dependency(typeof(LaunchActivity))]
namespace ShaPlay3.Droid
{
public class LaunchActivity : ILaunchActivity
{
public LaunchActivity()
{
}

public void LaunchActivityInAndroid()
{
var intent = new Intent(Forms.Context, typeof(ARinterface));
Forms.Context.StartActivity(intent);
}
}
}


However, the ARInterface activity has some differences.
first it needs to extend Activity instead of FormsAppCompatActivity, and then he has to override the OnCreate (Bundle bundle) method, inside the method he has to set the layout of the activity and then call the startAR method.


protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);

SetContentView(Resource.Layout.Main);

startAR();
}



Best reguards,

Dominic Jones
Thursday, March 4, 2021

Thank you,

After making those changes (and adding the Activity to the Manifest) it is up and running and I get Toasts when markers are found/lost - so definite progress!
I've pushed the latest version to Github so you can see the changes I made...
One note: your trial markers do not seem to work - it was only once I downloaded my own custom-generated marker that I got a Toast...
I'm now moving on to implementing the GL Renderer - is there anything that I need to be aware of/do differently on that side of things due to the changes made in implementing the first part (or that it's running under Forms)?
Finally, how do I now reference the member functions of the ARInterface class/instance, as I don't now have an instance of that in the Forms page code-behind (since it's launched by the ARLauncher class)? So, for instance, how would I call 'getMarkerID()'?
Many thanks once again - your help is invaluable, and hopefully these are the last queries before I have it fully working as needed...
Dom.

user profile image
Support Team  (expert)
Thursday, March 4, 2021

Hi,
you can now implement the renderer as described in the tutorial.

You cannot have an instance of the ARInterface because it is an activity and you cannot have multiple activities active at the same time.
That's why I thought about how to call certain functions from the ARScreen class of the Xamarin.Forms project.

The ARScreen class no longer calls the LaunchARActivityInAndroid function but defines an EventHandler that will be passed to the custom renderer.


[assembly: Xamarin.Forms.Dependency(typeof(ShaPlay3.IARinterface))]
namespace ShaPlay3.Views
{
[XamlCompilation(XamlCompilationOptions.Skip)] // Should be Skip?? Alt: Compile
public partial class ARScreen : ContentPage // This is where we should see the Pikkart AR activate
{

ILaunchARActivity ARengine;
public ARScreen()
{
InitializeComponent();
//ARengine = DependencyService.Get();
//ARengine.LaunchARActivityInAndroid();

testMarkerID();
}

private async void testMarkerID()
{
await Task.Delay(2000);

GetMarkerId();
}

public event EventHandler MarkerIdEvent;
public void GetMarkerId()
{
MarkerIdEvent?.Invoke(this, EventArgs.Empty);
}

private string _markerID;
public string MarkerID
{
set => _markerID = value;
}
}
}


The LaunchARActivity class is changed to an ARScreen custom renderer, OnElementChanged is overridden, and the ElementChangedEventArgs .NewElement parameter is cast to an ARScreen. This passes as the MarkerIdEvent event a method that will be defined.

This method now calls a static function of the ARinterface activity which returns a static variable containing the marker id


[assembly: ExportRenderer(typeof(ARScreen), typeof(LaunchARActivity))]
namespace ShaPlay3.Droid
{
public class LaunchARActivity : PageRenderer
{
private ARScreen _arScreen;

public LaunchARActivity(Context context) : base(context)
{

}

protected override void OnElementChanged(ElementChangedEventArgs e)
{
base.OnElementChanged(e);

if (e.NewElement != null)
{
_arScreen = (ARScreen)e.NewElement;

((ARScreen)e.NewElement).MarkerIdEvent += GetMarkerID;
}
var intent = new Intent(Forms.Context, typeof(ARinterface));
Forms.Context.StartActivity(intent);
}
private void GetMarkerID(object sender, EventArgs e)
{
string markerID = ARinterface.GetMarkerID();
_arScreen.MarkerID = markerID;
}
}
}


in the activity ARinterface create a copy of GetMarkerID but static so that it can be called from the custom renderer.


static string staticCurrentMarkerID = "";
public static string GetMarkerID()
{
return staticCurrentMarkerID;
}


Dominic Jones
Tuesday, March 9, 2021

Thank you, that's got me very close to where I need to be, I think...

Now have AR running on the ARScreen page, but still have a few issues/questions:

1) How do you "attach" a model to a marker? As the demo markers aren't working, I'm trying to get the Monkey model to appear on my custom Marker, but whilst I get a Toast saying it's found the model does not appear. Obviously in future I'd like to have many models attached to different markers, so would be good to understand how this works in a little detail... Feel free to point me at the relevant bits of your website/documentation if it's just something I've missed!

2) The Activity seems to start and stop when the screen orientation changes (portrait to landscape, etc) - is there a way to stop this happening and just have it keep running regardless? Or at least not reload the model and textures when it does?

3) Is there a way to stop the little "tracking point" hotspots from appearing on the render before a Marker is found? They only seem to appear until it starts tracking - they don't come back when it's lost - so hoping they can be turned off from the start?

4) I'm guessing that to remove the "Powered by PIKKART" logo at the bottom I will need to buy a licence, then it is automatically removed? Fine for now if so!

Many, many thanks once again...
Dom

user profile image
Support Team  (expert)
Tuesday, March 9, 2021

Hi,
we are glad our answers are helping you.

1 - The class that manages the rendering of 3D models and videos on the marker is the ARRenderer. The ARRenderer inside it initializes a mesh using the json of the 3D model and an image for the texture, then draws it in the onDrawFrame method. So you can add the logic with which to decide what to show.
Find the explanation and the classes I told you about at this address:
https://developer.pikkart.com/servizi/Menu/dinamica.aspx?idSezione=617&idArea=619&idCat=628&ID=1975&TipoElemento=page

2 - You can prevent the activity from reloading by putting android:configChanges="orientation|screenSize" in the tag.

3 - The effect you see is a choice of user experience that we have decided to adopt so that the user always has the impression that augmented reality is working. So no, it's not removable.

4 - The "Powered by PIKKART" logo is shown only in apps that use a trial license, once the license has been purchased and replaced, the logo no longer appears.

Best reguards,

Dominic Jones
Wednesday, March 10, 2021

Thanks - that all makes sense.

The problem for no (1) was self-inflicted - I had left some code in the MarkerFound() function that stopped the RecognitionFragment when it found a marker (that I'd put in as a test and left there by accident!), which explains why I couldn't work out why changes in ARRenderer were having no effect!

I think I'm all set from this point but will shout if I run into anything else - hopefully not!...

All the best, and many, many thanks once again...

Dom

Dominic Jones
Thursday, March 11, 2021

Sorry, I do have one more follow-up question:

I've been playing around with exiting the AR screen and restarting it, and have an odd issue:

As a simple test I've added an this.Finish(); command to the TrackingLost event handler, like this:

public void MarkerTrackingLost(string marker)
{
currentMarkerID = "";
currentlyTracking = false;
Toast.MakeText(this, "PikkartAR: lost tracking of marker " + marker, ToastLength.Short).Show();
this.Finish(); // Works once, but not again???!!
}

In the ARScreen onMarkerChanged() method I am launching a new SplashScreen so that I can then restart the AR component...

This all works fine the first cycle, but when I relaunch the AR for a second time it works perfectly, shows the test model, produces marker found/tracking lost toasts but the Finish() command does not end the Activity...

Any ideas why, or how I can fix this? I'll need to have the ARScreen being lanuched at various points for the demo project we're putting together, so important to be able to achieve this - I must be missing something simple, I'm sure...

Dominic Jones
Thursday, March 18, 2021

Just for the sake of anyone who stumbles across this thread and finds it useful, the solution to finishing the activity is to substitute:

this.Finish();

with:

var thisActivity = Platform.CurrentActivity;
thisActivity.Finish();

Hope that helps someone,
Dom

Sign in to add a comment