You have a mobile technology project. What tools do you need to test it?

The most important decision in testing is to define a clear objective on what to test for and how much to test. We need to have a clear predefined PASS/Fail criteria for each test. The test environment and tools etc can come later.

Don't jump into tool selection before charting your test course! Define your project's features, expected outcomes, performance benchmarks, and potential risks. This roadmap will guide you to the right testing types (functional, usability, etc.) and ensure you choose tools that align with your goals. Remember, clear objectives are the map to effective testing, not the anchor that weighs you down.

In the deployment of 5G SA networks, we leveraged network simulation tools like NS3 and NetSim to model network performance under various conditions. This helped in understanding the network's behavior in real-world scenarios before rollout. Utilizing Wireshark and tcpdump has been invaluable. These tools allowed us to capture and analyze traffic flows, identifying bottlenecks in both EPC and 5GS environments. Implementing automated testing frameworks, like Selenium for web interfaces and Appium for mobile applications may help the efficiency. These tools help facilitate continuous integration and continuous deployment (CI/CD) processes with no impact on the existing ecosystem.

Mobile app objective and vision has to be communicated with the QA Head/Testing team. Like features, functionalities and others. Create Test cases and scenarios. Check for performance under varied network (wifi, mobile network, switching between the both, usability, security points, etc) Use automated tools, simulators/emulators, however human testing would add value and will give good result. Testing from end user perspective should be done. Apart from identifying bugs(client, client/server), the UX should also be tracked and reported in some instance. App flow and consistency. Lag in the app? Test on variety of devices and OS, iOS, Android and any other. Check for in-app languages and consistency.

Mobile Testing is a software testing technique that evaluates mobile and desktop applications’ quality and functionality across different mobile devices, platforms, networks. It involves testing the application’s performance, usability, security, compatibility, other critical parameters that ensure an optimal user experience(UI). In order to build applications that are scalable and accessible across multiple platforms, automation testing can be used. As manual testing is time consuming, costly, and prone to errors, it makes sense to leverage automation for a wide range of repetitive tasks, such as regression and performance testing. Automated testing will allow the testers to quickly check errors, defects, and vulnerabilities.

Planning Effective Mobile Network Testing: Start with the Why: Clearly define your project's features, expected outcomes, and performance benchmarks. Identify potential risks and areas of concern. This planning map will guide you in choosing the right tools and tests. Go Beyond Speed: While Ookla Speedtest is popular, it only measures one aspect of performance. Consider tools offering a more comprehensive picture, including latency, jitter, packet loss, and signal strength. Know Your Users: Understand your customers' typical usage patterns and the devices they use. This guides you in selecting the right test devices and developing an effective testing strategy. Remember, usage environments can vary greatly.

Make sure to get feedback from the users you are creating the solution for. They will tell you everything you need to know regarding the viability and usability of your mobile project. Also, be open to other ways to deliver your solution that make your life easier.

The test environment is key to success! Depending on your target customers, the real world environment might be different. There might be products being used only in stationary environments like offices. There are products that are used on the go, while commuting. Make sure to learn about the usage patterns of your customers. Also, which devices are they using. If you have this knowledge, get the right test devices for your product and then make a testing strategy. What feature needs which test environment? How often do you want to test in mobile networks or in different physical environments?

Casi todos mis clientes intentan medir todos los eventos habidos y por haber. ¡ERROR! Es mucho mejor definir 10 eventos importantes y saber cómo medirlos y como interpretar los resultados. Empieza por 10 y ya tendrás tiempo de medir más. Otro tema suele ser la imperiosa necesidad de medir en tiempo real. ¿Para qué? ¿Vas a estar todo el día en la pantalla? No, ¿verdad? Pues interésate más por saber interpretar los datos y por un reporting sencillo antes que por ver puntos en una pantalla...

Evidentemente debemos buscar un entorno que sea lo más parecido al ambiente productivo futuro. Si existe conversion de datos dentro del proyecto, el ambiente ya debe contener estos datos. Si bien puede parecer imposible tener la conversion completa a etapas tempranas del proyecto, buscar un Golden Data que permitirá medir el comportamiento del ambiente dentro de las pruebas y podamos probar escenarios específicos dentro de las pruebas unitarias.

Picking the right testing tool isn't easy. Companies need to invest time and money to find the right testing tool that fits their needs and environments. There is no silver bullet tool out there. Every tool has its pros and cons. The challenge is to find the tool that ticks many of the required boxes. I usually recommend having a mix of testing tools available. For example, tools for unit testing, integration testing, API testing and to some extent UI testing tools. Depending on your tech stack, the tools will look different and also the amount of tests on each layer will differ. Cloud based tools can help to test more efficient on different devices at the same time.

New tools on the market such as our company’s Elemez offering provide real time intelligence and insights in device, app and network performance. This can be critical as changes are made and one wants real time insights into how the end user may be affected without depending on inbound help desk calls to alert you to problems. This is an emerging market segment but a critical part of any enterprise mobility toolkit.

Hay muchas herramientas para probar pero la principal es tu equipo. Un gran equipo de QA es lo más importante para poder diseñar y revisar pruebas. Después de esto, podrás escoger una buena herramienta; antes, nunca.

Siempre tener un emulador de pruebas será lo más fácil, aunque no siempre lo más económico, hay que tener en cuenta que habrá que adquirir las licencias y considerar los costos de implementación que van implícitos. Además utilizar herramientas de automatización de pruebas no consiste en solo adquirir el software, hay que tener consultores especializados que sepan "grabar" el paso a paso de las pruebas, deben ser consultores especialistas en la herramienta a probar, pero también especializados en la automatización de pruebas.

Nothing beats the actual real-life user testing. Nothing. All simulators are great, but they only good for testing repetitive tasks to ensure that nothing is broken after change

Finding the right testing method for your product is unique. Depending on the implemented or planned features, the methods will change. Teams should always consider a fair amount of time testing the main functionality of the product. Exploratory testing with session based testing can help to establish a lean approach to testing. In the wild, testing is also important to test against different environments and scenarios. Last but not the least hardware specific testing is important, too. Apps use many sensors and testers need to check that the processed data is correct. And one more thing, non-functional testing is equally important than checking just the main functionality. Nobody wants a bad UI/ UX. Nobody wants slow apps.

Testing is important not only to understand if the product/technology functions as planned but also to get a sense if it serves the needs of targeted users with the right features in place. For that reason the test users should be selected to represent the target group of users or the testing environment should simulate the pattern of target users if possible. The feedback of such a test would be valuable to fix the product with right features and functions in line with customer needs/asks and shape go to market strategy.

Some common methods categorized by their focus: Functional Testing: Unit Testing: Tests individual units of code (functions, classes) in isolation. Popular tools include JUnit, XCTest. Integration Testing: Verifies how different units function together. System Testing: Assesses the overall functionality of the app from a user perspective. Acceptance Testing: Confirms the app meets user requirements and business needs. User groups or beta testers are often involved. Non-Functional Testing: Performance Testing: Evaluates app speed, responsiveness, and resource usage under different loads. Compatibility Testing: Ensures the app works across different devices, operating systems, and screen sizes.

For enterprise apps, best practices include: 1. Regression testing 2. Limited deployment to a small range of representative devices 3. Deployment to a small % of the estate 4. Batched deployment. This phased approach limits the risks of bugs not caught during regression testing negatively affecting employees operating the new app or updated app in the rigors of the "real world". However, this phased approach is limited today. The only way the IT team in charge of deployment knows there is an issue is often by end-users opening trouble tickets. There is a better way, with new real-time Mobility Intelligence™ software from B2M Solutions (Elemez) and others that monitors devices' health in real-time to proactively find & fix bugs.

The development team should utilize the browser-based tools to assure the mobile experience is "pretty close". However, in my experience, having multiple phones (from different major carriers -AT&T, Verizon, etc), different OSs (iOS & Android) and multiple browsers (Chrome, Safari, Brave, etc) is the best way to test the mobile experience. This can be an investment the company makes, but if you're a small startup, it's probable that your team members personal phones will suffice). Test all entry point on every phone, OS, browser. Prioritize testing on: 1) Performance/speed; then 2) Usability * Lean towards speed; improve usability Capture all experience escapes by phone, carrier, etc. Remember: Solid + Shipped > "perfect"

Regardless of testing tools/suites, I always insist on testing on actual devices. Tools are great, but they never capture the nuisances of the actual mobile experience. I personally maintain a collection of older devices, that have intentionally NOT been upgraded to the latest operating systems (if possible). You can’t control what mobile devices a user might have, and these “artifacts” from mobile days gone by provide an excellent way to ensure legacy devices can access and use the system as well.

Testing on a wide range of devices and different OS versions is key. Likewise, if the app relies on integrations with other services, it is important to test those integrations. Finally, don't forget to have other groups within the company view and test the app, and collect feedback (including the legal department).

Think about the user experience you would like to deliver vs the capabilities of existing channels. Don't create limits based upon how you think existing channels work. Be open to other points of view and solutions that you might not have thought of. Remember, your mobile project is for your customers, and they don't care about how you did it - they only care about what it will do for them.

Un proyecto de automatización tiene incluidas estás tres partes: personas, tecnologías y procesos. Por lo que es necesario siempre nivelar el grado de participación de cada uno de estos componentes. Hay que recordar que siempre debemos priorizar el trato a las personas, que tengan un trato digno. La tecnología obviamente busca entre otras cosas, agilizar los procesos empresariales de tal forma que seamos más eficientes eliminando tareas redundantes o hiperoperativas. Pero las personas deben ser tratadas con dignidad y respeto ya que ellas son la fuente del conocimiento dentro de la empresa. Siempre considerar entonces opiniones de usuarios expertos en el diseño de la solución y la ejecución de pruebas.

The major goal of every CTO should be to decrease the number of upgrade cycles for the BASELINE program. That will free up the resources to enhance and establish the next baseline. The only way I found out how to do this is to stop relying on third-party frameworks. Your initial or subsequent releases will take longer, BUT that will remove your dependency on other upgrade cycles.